Paragon Software libère son pilote NTFS pour Linux avec une bien meilleure performance que le pilote actuel

Mais l'entreprise a du mal à intégrer les codes du développement collaboratif

Paragon Software libère son pilote NTFS pour Linux avec une bien meilleure performance que le pilote actuel,
mais l'entreprise a du mal à intégrer les codes du développement collaboratif

Manipuler une partition FAT ou FAT32 à partir d’une distribution Linux ne pose pas de problème : le montage, la lecture, l’écriture et le démontage se font sans accroc. C’est avec des supports de stockage en NTFS que le mur de difficultés peut se dresser. En fonction de la distribution Linux sur laquelle on s’appuie on peut être obligé de faire recours à des commandes Linux spécifiques. Motif : le support NTFS du célèbre noyau open source est limité. Paragon Software propose d’intégrer un correctif au noyau Linux pour apporter réponse à cette situation. Problème : l’éditeur soumet un patch de pilote NTFS dans un fichier de 27 000 lignes jugé difficile à exploiter par les mainteneurs.


Le pilote que Paragon Software propose permet un accès complet en lecture et en écriture sur les volumes NTFS et HFS+. En sus, il est doté d’utilitaires de gestion d’erreurs et de formatage, mais il s’agit de possibilités dont l’utilisateur dispose au travers d’une version payante. L’éditeur ne soumet que les fonctions de base accessibles sans cette dernière au processus de revue du noyau Linux. Celles-ci tiennent dans un fichier de 27 000 lignes de code dont une partie est proposée.

[CODE=C]
From 06f13dd3c9eb779d10a025f1d29a81d69e80412f Mon Sep 17 00:00:00 2001
From: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Date: Fri, 14 Aug 2020 12:24:35 +0300
Subject: [PATCH] fs: NTFS read-write driver GPL implementation by Paragon
Software.

This patch adds NTFS Read-Write driver to fs/ntfs3.

Having decades of expertise in commercial file systems development and huge
test coverage, we at Paragon Software GmbH want to make our contribution to
the Open Source Community by providing implementation of NTFS Read-Write
driver for the Linux Kernel.

This is fully functional NTFS Read-Write driver. Current version works with
NTFS(including v3.1) and normal/compressed/sparse files and supports journal replaying.

We plan to support this version after the codebase once merged, and add new
features and fix bugs. For example, full journaling support over JBD will be
added in later updates.

Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
---
Documentation/filesystems/ntfs3.rst | 83 +
MAINTAINERS | 7 +
fs/Kconfig | 1 +
fs/ntfs3/Kconfig | 23 +
fs/ntfs3/Makefile | 11 +
fs/ntfs3/attrib.c | 1277 +++++++
fs/ntfs3/attrlist.c | 455 +++
fs/ntfs3/bitfunc.c | 144 +
fs/ntfs3/bitmap.c | 1545 ++++++++
fs/ntfs3/debug.h | 77 +
fs/ntfs3/dir.c | 547 +++
fs/ntfs3/file.c | 1179 ++++++
fs/ntfs3/frecord.c | 2178 +++++++++++
fs/ntfs3/fslog.c | 5217 +++++++++++++++++++++++++++
fs/ntfs3/fsntfs.c | 2195 +++++++++++
fs/ntfs3/index.c | 2639 ++++++++++++++
fs/ntfs3/inode.c | 2068 +++++++++++
fs/ntfs3/lznt.c | 449 +++
fs/ntfs3/namei.c | 566 +++
fs/ntfs3/ntfs.h | 1253 +++++++
fs/ntfs3/ntfs_fs.h | 965 +++++
fs/ntfs3/record.c | 612 ++++
fs/ntfs3/run.c | 1188 ++++++
fs/ntfs3/super.c | 1412 ++++++++
fs/ntfs3/upcase.c | 78 +
fs/ntfs3/xattr.c | 968 +++++
26 files changed, 27137 insertions(+)
create mode 100644 Documentation/filesystems/ntfs3.rst
create mode 100644 fs/ntfs3/Kconfig
create mode 100644 fs/ntfs3/Makefile
create mode 100644 fs/ntfs3/attrib.c
create mode 100644 fs/ntfs3/attrlist.c
create mode 100644 fs/ntfs3/bitfunc.c
create mode 100644 fs/ntfs3/bitmap.c
create mode 100644 fs/ntfs3/debug.h
create mode 100644 fs/ntfs3/dir.c
create mode 100644 fs/ntfs3/file.c
create mode 100644 fs/ntfs3/frecord.c
create mode 100644 fs/ntfs3/fslog.c
create mode 100644 fs/ntfs3/fsntfs.c
create mode 100644 fs/ntfs3/index.c
create mode 100644 fs/ntfs3/inode.c
create mode 100644 fs/ntfs3/lznt.c
create mode 100644 fs/ntfs3/namei.c
create mode 100644 fs/ntfs3/ntfs.h
create mode 100644 fs/ntfs3/ntfs_fs.h
create mode 100644 fs/ntfs3/record.c
create mode 100644 fs/ntfs3/run.c
create mode 100644 fs/ntfs3/super.c
create mode 100644 fs/ntfs3/upcase.c
create mode 100644 fs/ntfs3/xattr.c

diff --git a/Documentation/filesystems/ntfs3.rst b/Documentation/filesystems/ntfs3.rst
new file mode 100644
index 000000000000..4f1398d74140
--- /dev/null
+++ b/Documentation/filesystems/ntfs3.rst
@@ -0,0 +1,83 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====
+NTFS3
+=====
+
+
+Summary and Features
+====================
+
+NTFS3 is fully functional NTFS Read-Write driver. The driver works with
+NTFS versions up to 3.1, normal/compressed/sparse files
+and journal replaying. File system type to use on mount is 'ntfs3'.
+
+- This driver implements NTFS read/write support for normal, sparsed and
+ compressed files.
+ NOTE: Operations with compressed files require increased memory consumption;
+- Supports native journal replaying;
+- Supports extended attributes;
+- Supports NFS export of mounted NTFS volumes.
+
+Mount Options
+=============
+
+The list below describes mount options supported by NTFS3 driver in addtion to
+generic ones.
+
+===============================================================================
+
+nls=name These options inform the driver how to interpret path
+ strings and translate them to Unicode and back. In case
+ none of these options are set, or if specified codepage
+ doesn't exist on the system, the default codepage will be
+ used (CONFIG_NLS_DEFAULT).
+ Examples:
+ 'nls=utf8'
+
+uid=
+gid=
+umask= Controls the default permissions for files/directories created
+ after the NTFS volume is mounted.
+
+fmask=
+dmask= Instead of specifying umask which applies both to
+ files and directories, fmask applies only to files and
+ dmask only to directories.
+
+nohidden Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
+ attribute will not be shown under Linux.
+
+sys_immutable Files with the Windows-specific SYSTEM
+ (FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
+ immutable files.
+
+discard Enable support of the TRIM command for improved performance
+ on delete operations, which is recommended for use with the
+ solid-state drives (SSD).
+
+force Forces the driver to mount partitions even if 'dirty' flag
+ (volume dirty) is set. Not recommended for use.
+
+sparse Create new files as "sparse".
+
+showmeta Use this parameter to show all meta-files (System Files) on
+ a mounted NTFS partition.
+ By default, all meta-files are hidden.
+
+======================= =======================================================
+
+
+ToDo list
+=========
+
+- Full journaling support (currently journal replaying is supported) over JBD.
+
+
+References
+==========
+https://www.paragon-software.com/home/ntfs-linux-professional/
+ - Commercial version of the NTFS driver for Linux.
+
+almaz.alexandrovich@paragon-software.com
+ - Direct e-mail address for feedback and requests on the NTFS3 implementation.
\ No newline at end of file
diff --git a/MAINTAINERS b/MAINTAINERS
index 2843effadff8..09b6f2aa36b9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12344,6 +12344,13 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/aia21/ntfs.git
F: Documentation/filesystems/ntfs.rst
F: fs/ntfs/

+NTFS3 FILESYSTEM
+M: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
+S: Supported
+W: http://www.paragon-software.com/
+F: Documentation/filesystems/ntfs3.rst
+F: fs/ntfs3/
+
NUBUS SUBSYSTEM
M: Finn Thain <fthain@telegraphics.com.au>
L: linux-m68k@lists.linux-m68k.org
diff --git a/fs/Kconfig b/fs/Kconfig
index aa4c12282301..eae96d55ab67 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -145,6 +145,7 @@ menu "DOS/FAT/EXFAT/NT Filesystems"
source "fs/fat/Kconfig"
source "fs/exfat/Kconfig"
source "fs/ntfs/Kconfig"
+source "fs/ntfs3/Kconfig"

endmenu
endif # BLOCK
diff --git a/fs/ntfs3/Kconfig b/fs/ntfs3/Kconfig
new file mode 100644
index 000000000000..92a9c68008c8
--- /dev/null
+++ b/fs/ntfs3/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NTFS3_FS
+ tristate "NTFS Read-Write file system support"
+ select NLS
+ help
+ Windows OS native file system (NTFS) support up to NTFS version 3.1.
+
+ Y or M enables the NTFS3 driver with full features enabled (read,
+ write, journal replaying, sparse/compressed files support).
+ File system type to use on mount is "ntfs3". Module name (M option)
+ is also "ntfs3".
+
+ Documentation: <fileocumentation/filesystems/ntfs3.rst>
+
+config NTFS3_64BIT_CLUSTER
+ bool "64 bits per NTFS clusters"
+ depends on NTFS3_FS && 64BIT
+ help
+ Windows implementation of ntfs.sys uses 32 bits per clusters.
+ If activated 64 bits per clusters you will be able to use 4k cluster
+ for 16T+ volumes. Windows will not be able to mount such volumes.
+
+ It is recommended to say N here.
diff --git a/fs/ntfs3/Makefile b/fs/ntfs3/Makefile
new file mode 100644
index 000000000000..4d4fe198b8b8
--- /dev/null
+++ b/fs/ntfs3/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the ntfs3 filesystem support.
+#
+
+obj-$(CONFIG_NTFS3_FS) += ntfs3.o
+
+ntfs3-objs := bitfunc.o bitmap.o inode.o fsntfs.o frecord.o \
+ index.o attrlist.o record.o attrib.o run.o xattr.o\
+ upcase.o super.o file.o dir.o namei.o lznt.o\
+ fslog.o

diff --git a/fs/ntfs3/attrib.c b/fs/ntfs3/attrib.c
new file mode 100644
index 000000000000..d61878b7b9e2
--- /dev/null
+++ b/fs/ntfs3/attrib.c
@@ -0,0 +1,1277 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * linux/fs/ntfs3/attrib.c
+ *
+ * Copyright (C) 2019-2020 Paragon Software GmbH, All rights reserved.
+ *
+ * TODO: merge attr_set_size/attr_data_get_block/attr_allocate_frame?
+ */
+

+#include <linux/blkdev.h>
+#include <linux/buffer_head.h>
+#include <linux/fs.h>
+#include <linux/hash.h>
+#include <linux/nls.h>
+#include <linux/ratelimit.h>
+#include <linux/sched/signal.h>
+#include <linux/slab.h>
+
+#include "debug.h"
+#include "ntfs.h"
+#include "ntfs_fs.h"
+
+#ifdef NTFS3_PREALLOCATE
+/*
+ * You can set external NTFS_MIN_LOG2_OF_CLUMP/NTFS_MAX_LOG2_OF_CLUMP to manage
+ * preallocate algorithm
+ */
+#ifndef NTFS_MIN_LOG2_OF_CLUMP
+#define NTFS_MIN_LOG2_OF_CLUMP 16
+#endif
+
+#ifndef NTFS_MAX_LOG2_OF_CLUMP
+#define NTFS_MAX_LOG2_OF_CLUMP 26
+#endif
+
+// 16M
+#define NTFS_CLUMP_MIN (1 << (NTFS_MIN_LOG2_OF_CLUMP + 8))
+// 16G
+#define NTFS_CLUMP_MAX (1ull << (NTFS_MAX_LOG2_OF_CLUMP + 8))
+
+/*
+ * get_pre_allocated
+ *
+ */
+static inline u64 get_pre_allocated(u64 size)
+{
+ u32 clump;
+ u8 align_shift;
+ u64 ret;
+
+ if (size <= NTFS_CLUMP_MIN) {
+ clump = 1 << NTFS_MIN_LOG2_OF_CLUMP;
+ align_shift = NTFS_MIN_LOG2_OF_CLUMP;
+ } else if (size >= NTFS_CLUMP_MAX) {
+ clump = 1 << NTFS_MAX_LOG2_OF_CLUMP;
+ align_shift = NTFS_MAX_LOG2_OF_CLUMP;
+ } else {
+ align_shift = NTFS_MIN_LOG2_OF_CLUMP - 1 +
+ __ffs(size >> (8 + NTFS_MIN_LOG2_OF_CLUMP));
+ clump = 1u << align_shift;
+ }
+
+ ret = (((size + clump - 1) >> align_shift)) << align_shift;
+
+ return ret;
+}
+#endif
+
+/*
+ * attr_must_be_resident
+ *
+ * returns true if attribute must be resident
+ */
+static inline bool attr_must_be_resident(ntfs_sb_info *sbi, ATTR_TYPE type)
+{
+ const ATTR_DEF_ENTRY *de;
+
+ switch (type) {
+ case ATTR_STD:
+ case ATTR_NAME:
+ case ATTR_ID:
+ case ATTR_LABEL:
+ case ATTR_VOL_INFO:
+ case ATTR_ROOT:
+ case ATTR_EA_INFO:
+ return true;
+ default:
+ de = ntfs_query_def(sbi, type);
+ if (de && (de->flags & NTFS_ATTR_MUST_BE_RESIDENT))
+ return true;
+ return false;
+ }
+}
+
+/*
+ * attr_load_runs
+ *
+ * load all runs stored in 'attr'
+ */
+int attr_load_runs(ATTRIB *attr, ntfs_inode *ni, struct runs_tree *run)
+{
+ int err;
+ CLST svcn = le64_to_cpu(attr->nres.svcn);
+ CLST evcn = le64_to_cpu(attr->nres.evcn);
+ u32 asize;
+ u16 run_off;
+
+ if (svcn >= evcn + 1 || run_is_mapped_full(run, svcn, evcn))
+ return 0;
+
+ asize = le32_to_cpu(attr->size);
+ run_off = le16_to_cpu(attr->nres.run_off);
+ err = run_unpack_ex(run, ni->mi.sbi, ni->mi.rno, svcn, evcn,
+ Add2Ptr(attr, run_off), asize - run_off);
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
+/*
+ * int run_deallocate_ex
+ *
+ * Deallocate clusters
+ */
+static int run_deallocate_ex(ntfs_sb_info *sbi, struct runs_tree *run, CLST vcn,
+ CLST len, CLST *done, bool trim)
+{
+ int err = 0;
+ CLST vcn0 = vcn, lcn, clen, dn = 0;
+ size_t idx;
+
+ if (!len)
+ goto out;
+
+ if (!run_lookup_entry(run, vcn, &lcn, &clen, &idx)) {
+failed:
+ run_truncate(run, vcn0);
+ err = -EINVAL;
+ goto out;
+ }
+
+ for (; {
+ if (clen > len)
+ clen = len;
+
+ if (!clen) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ if (lcn != SPARSE_LCN) {
+ mark_as_free_ex(sbi, lcn, clen, trim);
+ dn += clen;
+ }
+
+ len -= clen;
+ if (!len)
+ break;
+
+ if (!run_get_entry(run, ++idx, &vcn, &lcn, &clen)) {
+ // save memory - don't load entire run
+ goto failed;
+ }
+ }
+
+out:
+ if (done)
+ *done = dn;
+
+ return err;
+}
+
+/*
+ * attr_allocate_clusters
+ *
+ * find free space, mark it as used and store in 'run'
+ */
+int attr_allocate_clusters(ntfs_sb_info *sbi, struct runs_tree *run, CLST vcn,
+ CLST lcn, CLST len, CLST *pre_alloc,
+ enum ALLOCATE_OPT opt, CLST *alen, const size_t fr,
+ CLST *new_lcn)
+{
+ int err;
+ CLST flen, vcn0 = vcn, pre = pre_alloc ? *pre_alloc : 0;
+ wnd_bitmap *wnd = &sbi->used.bitmap;
+ size_t cnt = run->count;
+
+ for (; {
+ err = ntfs_look_for_free_space(sbi, lcn, len + pre, &lcn, &flen,
+ opt);
+
+#ifdef NTFS3_PREALLOCATE
+ if (err == -ENOSPC && pre) {
+ pre = 0;
+ if (*pre_alloc)
+ *pre_alloc = 0;
+ continue;
+ }
+#endif
+
+ if (err)
+ goto out;
+
+ if (new_lcn && vcn == vcn0)
+ *new_lcn = lcn;
+
+ /* Add new fragment into run storage */
+ if (!run_add_entry(run, vcn, lcn, flen)) {
+ down_write_nested(&wnd->rw_lock, BITMAP_MUTEX_CLUSTERS);
+ wnd_set_free(wnd, lcn, flen);
+ up_write(&wnd->rw_lock);
+ err = -ENOMEM;
+ goto out;
+ }
+
+ vcn += flen;
+
+ if (flen >= len || opt == ALLOCATE_MFT ||
+ (fr && run->count - cnt >= fr)) {
+ *alen = vcn - vcn0;
+ return 0;
+ }
+
+ len -= flen;
+ }
+
+out:
+ /* undo */
+ run_deallocate_ex(sbi, run, vcn0, vcn - vcn0, NULL, false);
+ run_truncate(run, vcn0);
+
+ return err;
+}
+
+/*
+ * attr_set_size_res
+ *
+ * helper for attr_set_size
+ */
+static int attr_set_size_res(ntfs_inode *ni, ATTRIB *attr, u64 new_size,
+ struct runs_tree *run, ATTRIB **ins_attr)
+{
+ int err = 0;
+ mft_inode *mi = &ni->mi;
+ ntfs_sb_info *sbi = mi->sbi;
+ MFT_REC *rec = mi->mrec;
+ u32 used = le32_to_cpu(rec->used);
+ u32 asize = le32_to_cpu(attr->size);
+ u32 aoff = PtrOffset(rec, attr);
+ u32 rsize = le32_to_cpu(attr->res.data_size);
+ u32 tail = used - aoff - asize;
+ char *next = Add2Ptr(attr, asize);
+ int dsize;
+ CLST len, alen;
+ ATTRIB *attr_s = NULL;
+ bool is_ext;
+
+ if (new_size >= sbi->max_bytes_per_attr)
+ goto resident2nonresident;
+
+ dsize = QuadAlign(new_size) - QuadAlign(rsize);
+
+ if (dsize < 0) {
+ memmove(next + dsize, next, tail);
+ } else if (dsize > 0) {
+ if (used + dsize > sbi->max_bytes_per_attr)
+ goto resident2nonresident;
+ memmove(next + dsize, next, tail);
+ memset(next, 0, dsize);
+ }
+
+ rec->used = cpu_to_le32(used + dsize);
+ attr->size = cpu_to_le32(asize + dsize);
+ attr->res.data_size = cpu_to_le32(new_size);
+ mi->dirty = true;
+ *ins_attr = attr;
+
+ return 0;
+
+resident2nonresident:
+ len = bytes_to_cluster(sbi, rsize);
+
+ run_init(run);
+
+ is_ext = is_attr_ext(attr);
+
+ if (!len) {
+ alen = 0;
+ } else if (is_ext) {
+ if (!run_add_entry(run, 0, SPARSE_LCN, len)) {
+ err = -ENOMEM;
+ goto out;
+ }
+ alen = len;
+ } else {
+ err = attr_allocate_clusters(sbi, run, 0, 0, len, NULL,
+ ALLOCATE_DEF, &alen, 0, NULL);
+ if (err)
+ goto out;
+
+ err = ntfs_sb_write_run(sbi, run, 0, resident_data(attr),
+ rsize);
+ if (err)
+ goto out;
+ }
+
+ attr_s = ntfs_memdup(attr, asize);
+ if (!attr_s) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ /*verify(mi_remove_attr(mi, attr));*/
+ used -= asize;
+ memmove(attr, Add2Ptr(attr, asize), used - aoff);
+ rec->used = cpu_to_le32(used);
+ mi->dirty = true;
+
+ err = ni_insert_nonresident(ni, attr_s->type, attr_name(attr_s),
+ attr_s->name_len, run, 0, alen,
+ attr_s->flags, &attr, NULL);
+ if (err)
+ goto out;
+
+ ntfs_free(attr_s);
+ attr->nres.data_size = cpu_to_le64(rsize);
+ attr->nres.valid_size = attr->nres.data_size;
+
+ *ins_attr = attr;
+
+ if (attr_s->type == ATTR_DATA && !attr_s->name_len &&
+ run == &ni->file.run) {
+ ni->ni_flags &= ~NI_FLAG_RESIDENT;
+ }
+
+ /* Resident attribute becomes non resident */
+ return 0;
+
+out:
+ /* undo: do not trim new allocated clusters */
+ run_deallocate(sbi, run, false);
+ run_close(run);
+
+ if (attr_s) {
+ memmove(next, Add2Ptr(rec, aoff), used - aoff);
+ memcpy(Add2Ptr(rec, aoff), attr_s, asize);
+ rec->used = cpu_to_le32(used + asize);
+ mi->dirty = true;
+ ntfs_free(attr_s);
+ }
+
+ return err;
+}
+
+/*
+ * attr_set_size
+ *
+ * change the size of attribute
+ * Extend:
+ * - sparse/compressed: no allocated clusters
+ * - normal: append allocated and preallocated new clusters
+ * Shrink:
+ * - no deallocate if keep_prealloc is set
+ */
+int attr_set_size(ntfs_inode *ni, ATTR_TYPE type, const __le16 *name,
+ u8 name_len, struct runs_tree *run, u64 new_size,
+ const u64 *new_valid, bool keep_prealloc, ATTRIB **ret)
+{
+ int err = 0;
+ ntfs_sb_info *sbi = ni->mi.sbi;
+ u8 cluster_bits = sbi->cluster_bits;
+ bool is_mft =
+ ni->mi.rno == MFT_REC_MFT && type == ATTR_DATA && !name_len;
+ u64 old_valid, old_size, old_alloc, new_alloc, new_alloc_tmp;
+ ATTRIB *attr, *attr_b;
+ ATTR_LIST_ENTRY *le, *le_b;
+ mft_inode *mi, *mi_b;
+ CLST alen, vcn, lcn, new_alen, old_alen, svcn, evcn;
+ CLST next_svcn, pre_alloc = -1, done = 0;
+ bool is_ext;
+ u32 align;
+ MFT_REC *rec;
+
+again:
+ le_b = NULL;
+ attr_b = ni_find_attr(ni, NULL, &le_b, type, name, name_len, NULL,
+ &mi_b);
+ if (!attr_b) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ if (!attr_b->non_res) {
+ err = attr_set_size_res(ni, attr_b, new_size, run, &attr_b);
+ if (err)
+ goto out;
+
+ if (!attr_b->non_res)
+ goto out;
+
+ /* Resident attribute becomes non resident */
+ goto again;
+ }
+
+ is_ext = is_attr_ext(attr_b);
+
+again_1:
+
+ if (is_ext) {
+ align = 1u << (attr_b->nres.c_unit + cluster_bits);
+ if (is_attr_sparsed(attr_b))
+ keep_prealloc = false;
+ } else {
+ align = sbi->cluster_size;
+ }
+
+ old_valid = le64_to_cpu(attr_b->nres.valid_size);
+ old_size = le64_to_cpu(attr_b->nres.data_size);
+ old_alloc = le64_to_cpu(attr_b->nres.alloc_size);
+ old_alen = old_alloc >> cluster_bits;
+
+ new_alloc = (new_size + align - 1) & ~(u64)(align - 1);
+ new_alen = new_alloc >> cluster_bits;
+
+ if (keep_prealloc && is_ext)
+ keep_prealloc = false;
+
+ if (keep_prealloc && new_size < old_size) {
+ attr_b->nres.data_size = cpu_to_le64(new_size);
+ mi_b->dirty = true;
+ goto ok;
+ }
+
+ vcn = old_alen - 1;
+
+ svcn = le64_to_cpu(attr_b->nres.svcn);
+ evcn = le64_to_cpu(attr_b->nres.evcn);
+
+ if (svcn <= vcn && vcn <= evcn) {
+ attr = attr_b;
+ le = le_b;
+ mi = mi_b;
+ } else if (!le_b) {
+ err = -EINVAL;
+ goto out;
+ } else {
+ le = le_b;
+ attr = ni_find_attr(ni, attr_b, &le, type, name, name_len, &vcn,
+ &mi);
+ if (!attr) {
+ err = -EINVAL;
+ goto out;
+ }
+
+next_le_1:
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn = le64_to_cpu(attr->nres.evcn);
+ }
+
+next_le:
+ rec = mi->mrec;
+
+ err = attr_load_runs(attr, ni, run);
+ if (err)
+ goto out;
+
+ if (new_size > old_size) {
+ CLST to_allocate;
+ size_t cnt, free;
+
+ if (new_alloc <= old_alloc) {
+ attr_b->nres.data_size = cpu_to_le64(new_size);
+ mi_b->dirty = true;
+ goto ok;
+ }
+
+ to_allocate = new_alen - old_alen;
+add_alloc_in_same_attr_seg:
+ lcn = 0;
+ if (is_mft) {
+ /* mft allocates clusters from mftzone */
+ pre_alloc = 0;
+ } else if (is_ext) {
+ /* no preallocate for sparse/compress */
+ pre_alloc = 0;
+ } else if (pre_alloc == -1) {
+ pre_alloc = 0;
+#ifdef NTFS3_PREALLOCATE
+ if (type == ATTR_DATA && !name_len) {
+ CLST new_alen2 = bytes_to_cluster(
+ sbi, get_pre_allocated(new_size));
+ pre_alloc = new_alen2 - new_alen;
+ }
+#endif
+ /* Get the last lcn to allocate from */
+ if (old_alen &&
+ !run_lookup_entry(run, vcn, &lcn, NULL, NULL)) {
+ lcn = SPARSE_LCN;
+ }
+
+ if (lcn == SPARSE_LCN)
+ lcn = 0;
+ else if (lcn)
+ lcn += 1;
+
+ free = wnd_zeroes(&sbi->used.bitmap);
+ if (to_allocate > free) {
+ err = -ENOSPC;
+ goto out;
+ }
+
+ if (pre_alloc && to_allocate + pre_alloc > free)
+ pre_alloc = 0;
+ }
+
+ vcn = old_alen;
+ cnt = run->count;
+
+ if (is_ext) {
+ if (!run_add_entry(run, vcn, SPARSE_LCN, to_allocate)) {
+ err = -ENOMEM;
+ goto out;
+ }
+ alen = to_allocate;
+ } else {
+ /* ~3 bytes per fragment */
+ err = attr_allocate_clusters(
+ sbi, run, vcn, lcn, to_allocate, &pre_alloc,
+ is_mft ? ALLOCATE_MFT : 0, &alen,
+ is_mft ? 0 :
+ (sbi->record_size -
+ le32_to_cpu(rec->used) + 8) /
+ 3 +
+ 1,
+ NULL);
+ if (err)
+ goto out;
+ }
+
+ done += alen;
+ vcn += alen;
+ if (to_allocate > alen)
+ to_allocate -= alen;
+ else
+ to_allocate = 0;
+
+pack_runs:
+ err = mi_pack_runs(mi, attr, run, vcn - svcn);
+ if (err)
+ goto out;
+
+ next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
+ new_alloc_tmp = (u64)next_svcn << cluster_bits;
+ attr_b->nres.alloc_size = cpu_to_le64(new_alloc_tmp);
+ mi_b->dirty = true;
+
+ if (next_svcn >= vcn && !to_allocate) {
+ /* Normal way. update attribute and exit */
+ attr_b->nres.data_size = cpu_to_le64(new_size);
+ goto ok;
+ }
+
+ /* at least two mft to avoid recursive loop*/
+ if (is_mft && next_svcn == vcn &&
+ (done << sbi->cluster_bits) >= 2 * sbi->record_size) {
+ new_size = new_alloc_tmp;
+ attr_b->nres.data_size = attr_b->nres.alloc_size;
+ goto ok;
+ }
+
+ if (le32_to_cpu(rec->used) < sbi->record_size) {
+ old_alen = next_svcn;
+ evcn = old_alen - 1;
+ goto add_alloc_in_same_attr_seg;
+ }
+
+ if (type == ATTR_LIST) {
+ err = ni_expand_list(ni);
+ if (err)
+ goto out;
+ if (next_svcn < vcn)
+ goto pack_runs;
+
+ /* layout of records is changed */
+ goto again;
+ }
+
+ if (!ni->attr_list.size) {
+ err = ni_create_attr_list(ni);
+ if (err)
+ goto out;
+ /* layout of records is changed */
+ }
+
+ if (next_svcn >= vcn)
+ goto again;
+
+ /* insert new attribute segment */
+ err = ni_insert_nonresident(ni, type, name, name_len, run,
+ next_svcn, vcn - next_svcn,
+ attr_b->flags, &attr, &mi);
+ if (err)
+ goto out;
+
+ if (!is_mft)
+ run_truncate_head(run, evcn + 1);
+
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn = le64_to_cpu(attr->nres.evcn);
+
+ le_b = NULL;
+ /* layout of records maybe changed */
+ /* find base attribute to update*/
+ attr_b = ni_find_attr(ni, NULL, &le_b, type, name, name_len,
+ NULL, &mi_b);
+ if (!attr_b) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ attr_b->nres.alloc_size = cpu_to_le64(vcn << cluster_bits);
+ attr_b->nres.data_size = attr_b->nres.alloc_size;
+ attr_b->nres.valid_size = attr_b->nres.alloc_size;
+ mi_b->dirty = true;
+ goto again_1;
+ }
+
+ if (new_size != old_size ||
+ (new_alloc != old_alloc && !keep_prealloc)) {
+ vcn = max(svcn, new_alen);
+ new_alloc_tmp = (u64)vcn << cluster_bits;
+
+ err = run_deallocate_ex(sbi, run, vcn, evcn - vcn + 1, &alen,
+ true);
+ if (err)
+ goto out;
+
+ run_truncate(run, vcn);
+
+ if (vcn > svcn) {
+ err = mi_pack_runs(mi, attr, run, vcn - svcn);
+ if (err < 0)
+ goto out;
+ } else if (le && le->vcn) {
+ u16 le_sz = le16_to_cpu(le->size);
+
+ /*
+ * NOTE: list entries for one attribute are always
+ * the same size. We deal with last entry (vcn==0)
+ * and it is not first in entries array
+ * (list entry for std attribute always first)
+ * So it is safe to step back
+ */
+ mi_remove_attr(mi, attr);
+
+ if (!al_remove_le(ni, le)) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ le = (ATTR_LIST_ENTRY *)((u8 *)le - le_sz);
+ } else {
+ attr->nres.evcn = cpu_to_le64((u64)vcn - 1);
+ mi->dirty = true;
+ }
+
+ attr_b->nres.alloc_size = cpu_to_le64(new_alloc_tmp);
+
+ if (vcn == new_alen) {
+ attr_b->nres.data_size = cpu_to_le64(new_size);
+ if (new_size < old_valid)
+ attr_b->nres.valid_size =
+ attr_b->nres.data_size;
+ } else {
+ if (new_alloc_tmp <=
+ le64_to_cpu(attr_b->nres.data_size))
+ attr_b->nres.data_size =
+ attr_b->nres.alloc_size;
+ if (new_alloc_tmp <
+ le64_to_cpu(attr_b->nres.valid_size))
+ attr_b->nres.valid_size =
+ attr_b->nres.alloc_size;
+ }
+
+ if (is_ext)
+ le64_sub_cpu(&attr_b->nres.total_size,
+ ((u64)alen << cluster_bits));
+
+ mi_b->dirty = true;
+
+ if (new_alloc_tmp <= new_alloc)
+ goto ok;
+
+ old_size = new_alloc_tmp;
+ vcn = svcn - 1;
+
+ if (le == le_b) {
+ attr = attr_b;
+ mi = mi_b;
+ evcn = svcn - 1;
+ svcn = 0;
+ goto next_le;
+ }
+
+ if (le->type != type || le->name_len != name_len ||
+ memcmp(le_name(le), name, name_len * sizeof(short))) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ err = ni_load_mi(ni, le, &mi);
+ if (err)
+ goto out;
+
+ attr = mi_find_attr(mi, NULL, type, name, name_len, &le->id);
+ if (!attr) {
+ err = -EINVAL;
+ goto out;
+ }
+ goto next_le_1;
+ }
+
+ok:
+ if (new_valid) {
+ __le64 valid = cpu_to_le64(min(*new_valid, new_size));
+
+ if (attr_b->nres.valid_size != valid) {
+ attr_b->nres.valid_size = valid;
+ mi_b->dirty = true;
+ }
+ }
+
+out:
+ if (!err && attr_b && ret)
+ *ret = attr_b;
+
+ /* update inode_set_bytes*/
+ if (!err && attr_b && attr_b->non_res &&
+ ((type == ATTR_DATA && !name_len) ||
+ (type == ATTR_ALLOC && name == I30_NAME))) {
+ ni->vfs_inode.i_size = new_size;
+ inode_set_bytes(&ni->vfs_inode,
+ le64_to_cpu(attr_b->nres.alloc_size));
+ }
+
+ return err;
+}
+
+int attr_data_get_block(ntfs_inode *ni, CLST vcn, CLST *lcn, CLST *len,
+ bool *new)
+{
+ int err = 0;
+ struct runs_tree *run = &ni->file.run;
+ ntfs_sb_info *sbi;
+ u8 cluster_bits;
+ ATTRIB *attr, *attr_b;
+ ATTR_LIST_ENTRY *le, *le_b;
+ mft_inode *mi, *mi_b;
+ CLST hint, svcn, evcn1, new_evcn1, next_svcn;
+ u64 new_size, total_size, new_alloc;
+ u32 clst_per_frame, frame_size;
+ bool ok;
+
+ if (new)
+ *new = false;
+
+ down_read(&ni->file.run_lock);
+ ok = run_lookup_entry(run, vcn, lcn, len, NULL);
+ up_read(&ni->file.run_lock);
+
+ if (ok && (*lcn != SPARSE_LCN || !new)) {
+ /* normal way */
+ return 0;
+ }
+
+ sbi = ni->mi.sbi;
+ cluster_bits = sbi->cluster_bits;
+ new_size = ((u64)vcn + 1) << cluster_bits;
+
+ ni_lock(ni);
+ down_write(&ni->file.run_lock);
+
+again:
+ le_b = NULL;
+ attr_b = ni_find_attr(ni, NULL, &le_b, ATTR_DATA, NULL, 0, NULL, &mi_b);
+ if (!attr_b) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ if (!attr_b->non_res) {
+ if (!new) {
+ *lcn = RESIDENT_LCN;
+ goto out;
+ }
+
+ err = attr_set_size_res(ni, attr_b, new_size, run, &attr_b);
+ if (err)
+ goto out;
+
+ if (!attr_b->non_res) {
+ /* Resident attribute still resident */
+ *lcn = RESIDENT_LCN;
+ goto out;
+ }
+
+ /* Resident attribute becomes non resident */
+ goto again;
+ }
+
+ clst_per_frame = 1u << attr_b->nres.c_unit;
+ frame_size = clst_per_frame << cluster_bits;
+ new_alloc = (new_size + frame_size - 1) & ~(u64)(frame_size - 1);
+
+ svcn = le64_to_cpu(attr_b->nres.svcn);
+ evcn1 = le64_to_cpu(attr_b->nres.evcn) + 1;
+
+ attr = attr_b;
+ le = le_b;
+ mi = mi_b;
+
+ if (le_b && (vcn < svcn || evcn1 <= vcn)) {
+ attr = ni_find_attr(ni, attr_b, &le, ATTR_DATA, NULL, 0, &vcn,
+ &mi);
+ if (!attr) {
+ err = -EINVAL;
+ goto out;
+ }
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn1 = le64_to_cpu(attr->nres.evcn) + 1;
+ }
+
+ err = attr_load_runs(attr, ni, run);
+ if (err)
+ goto out;
+
+ if (!ok) {
+ ok = run_lookup_entry(run, vcn, lcn, len, NULL);
+ if (ok && (*lcn != SPARSE_LCN || !new)) {
+ /* normal way */
+ err = 0;
+ goto out;
+ }
+ }
+
+ if (!is_attr_ext(attr_b)) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ /* Get the last lcn to allocate from */
+ hint = 0;
+
+ if (vcn > evcn1) {
+ if (!run_add_entry(run, evcn1, SPARSE_LCN, vcn - evcn1)) {
+ err = -ENOMEM;
+ goto out;
+ }
+ } else if (vcn && !run_lookup_entry(run, vcn - 1, &hint, NULL, NULL)) {
+ hint = -1;
+ }
+
+ err = attr_allocate_clusters(sbi, run, vcn, hint + 1, clst_per_frame,
+ NULL, 0, len, 0, lcn);
+ if (err)
+ goto out;
+
+ *new = true;
+
+ new_evcn1 = vcn + clst_per_frame;
+ if (new_evcn1 < evcn1)
+ new_evcn1 = evcn1;
+
+ total_size = le64_to_cpu(attr_b->nres.total_size) + frame_size;
+
+repack:
+
+ err = mi_pack_runs(mi, attr, run, new_evcn1 - svcn);
+ if (err < 0)
+ goto out;
+
+ attr_b->nres.total_size = cpu_to_le64(total_size);
+ inode_set_bytes(&ni->vfs_inode, total_size);
+
+ mi_b->dirty = true;
+ mark_inode_dirty(&ni->vfs_inode);
+
+ next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
+
+ if (next_svcn >= evcn1) {
+ /* Normal way. update attribute and exit */
+ goto out;
+ }
+
+ if (!ni->attr_list.le) {
+ err = ni_create_attr_list(ni);
+ if (err)
+ goto out;
+ /* layout of records is changed */
+ le_b = NULL;
+ attr_b = ni_find_attr(ni, NULL, &le_b, ATTR_DATA, NULL, 0, NULL,
+ &mi_b);
+ if (!attr_b) {
+ err = -ENOENT;
+ goto out;
+ }
+
+ attr = attr_b;
+ le = le_b;
+ mi = mi_b;
+ goto repack;
+ }
+
+ /* Estimate next attribute */
+ attr = ni_find_attr(ni, attr, &le, ATTR_DATA, NULL, 0, &evcn1, &mi);
+
+ if (attr && le32_to_cpu(mi->mrec->used) + 8 <= sbi->record_size) {
+ svcn = next_svcn;
+ evcn1 = le64_to_cpu(attr->nres.evcn) + 1;
+
+ err = attr_load_runs(attr, ni, run);
+ if (err)
+ goto out;
+
+ attr->nres.svcn = cpu_to_le64(svcn);
+ err = mi_pack_runs(mi, attr, run, evcn1 - svcn);
+ if (err < 0)
+ goto out;
+
+ le->vcn = cpu_to_le64(svcn);
+
+ mi->dirty = true;
+
+ next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
+
+ if (next_svcn >= evcn1) {
+ /* Normal way. update attribute and exit */
+ goto out;
+ }
+ }
+
+ err = ni_insert_nonresident(ni, ATTR_DATA, NULL, 0, run, next_svcn,
+ evcn1 - next_svcn, attr_b->flags, &attr,
+ &mi);
+ if (err)
+ goto out;
+
+ run_truncate_head(run, vcn);
+
+out:
+ up_write(&ni->file.run_lock);
+ ni_unlock(ni);
+
+ return err;
+}
+
+/*
+ * attr_load_runs_vcn
+ *
+ * load runs with vcn
+ */
+int attr_load_runs_vcn(ntfs_inode *ni, ATTR_TYPE type, const __le16 *name,
+ u8 name_len, struct runs_tree *run, CLST vcn)
+{
+ ATTRIB *attr;
+ int err;
+ CLST svcn, evcn;
+ u16 ro;
+
+ attr = ni_find_attr(ni, NULL, NULL, type, name, name_len, &vcn, NULL);
+ if (!attr)
+ return -ENOENT;
+
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn = le64_to_cpu(attr->nres.evcn);
+
+ if (evcn < vcn || vcn < svcn)
+ return -EINVAL;
+
+ ro = le16_to_cpu(attr->nres.run_off);
+ err = run_unpack_ex(run, ni->mi.sbi, ni->mi.rno, svcn, evcn,
+ Add2Ptr(attr, ro), le32_to_cpu(attr->size) - ro);
+ if (err < 0)
+ return err;
+ return 0;
+}
+
+/*
+ * attr_is_frame_compressed
+ *
+ * This function is used to detect compressed frame
+ */
+int attr_is_frame_compressed(ntfs_inode *ni, ATTRIB *attr, CLST frame,
+ CLST *clst_data, bool *is_compr)
+{
+ int err;
+ u32 clst_frame;
+ CLST len, lcn, vcn, alen, slen, vcn1;
+ size_t idx;
+ struct runs_tree *run;
+
+ *clst_data = 0;
+ *is_compr = false;
+
+ if (!is_attr_compressed(attr))
+ return 0;
+
+ if (!attr->non_res)
+ return 0;
+
+ clst_frame = 1u << attr->nres.c_unit;
+ vcn = frame * clst_frame;
+ run = &ni->file.run;
+
+ if (!run_lookup_entry(run, vcn, &lcn, &len, &idx)) {
+ err = attr_load_runs_vcn(ni, attr->type, attr_name(attr),
+ attr->name_len, run, vcn);
+ if (err)
+ return err;
+
+ if (!run_lookup_entry(run, vcn, &lcn, &len, &idx))
+ return -ENOENT;
+ }
+
+ if (lcn == SPARSE_LCN) {
+ /* The frame is sparsed if "clst_frame" clusters are sparsed */
+ *is_compr = true;
+ return 0;
+ }
+
+ if (len >= clst_frame) {
+ /*
+ * The frame is not compressed 'cause
+ * it does not contain any sparse clusters
+ */
+ *clst_data = clst_frame;
+ return 0;
+ }
+
+ alen = bytes_to_cluster(ni->mi.sbi, le64_to_cpu(attr->nres.alloc_size));
+ slen = 0;
+ *clst_data = len;
+
+ /*
+ * The frame is compressed if *clst_data + slen >= clst_frame
+ * Check next fragments
+ */
+ while ((vcn += len) < alen) {
+ vcn1 = vcn;
+
+ if (!run_get_entry(run, ++idx, &vcn, &lcn, &len) ||
+ vcn1 != vcn) {
+ err = attr_load_runs_vcn(ni, attr->type,
+[/patch]...