mirror of
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2025-05-24 10:39:52 +00:00
39d42fa96b
This is a follow up to [1] that detailed latency problems associated with dm-crypt's use of workqueues when processing IO. Current dm-crypt implementation creates a significant IO performance overhead (at least on small IO block sizes) for both latency and throughput. We suspect offloading IO request processing into workqueues and async threads is more harmful these days with the modern fast storage. I also did some digging into the dm-crypt git history and much of this async processing is not needed anymore, because the reasons it was added are mostly gone from the kernel. More details can be found in [2] (see "Git archeology" section). This change adds DM_CRYPT_NO_READ_WORKQUEUE and DM_CRYPT_NO_WRITE_WORKQUEUE flags for read and write BIOs, which direct dm-crypt to not offload crypto operations into kcryptd workqueues. In addition, writes are not buffered to be sorted in the dm-crypt red-black tree, but dispatched immediately. For cases, where crypto operations cannot happen (hard interrupt context, for example the read path of some NVME drivers), we offload the work to a tasklet rather than a workqueue. These flags only ensure no async BIO processing in the dm-crypt module. It is worth noting that some Crypto API implementations may offload encryption into their own workqueues, which are independent of the dm-crypt and its configuration. However upon enabling these new flags dm-crypt will instruct Crypto API not to backlog crypto requests. To give an idea of the performance gains for certain workloads, consider the script, and results when tested against various devices, detailed here: https://www.redhat.com/archives/dm-devel/2020-July/msg00138.html [1]: https://www.spinics.net/lists/dm-crypt/msg07516.html [2]: https://blog.cloudflare.com/speeding-up-linux-disk-encryption/ Signed-off-by: Ignat Korchagin <ignat@cloudflare.com> Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Bob Liu <bob.liu@oracle.com> Signed-off-by: Mike Snitzer <snitzer@redhat.com> |
||
|---|---|---|
| .. | ||
| bcache | ||
| persistent-data | ||
| dm-bio-prison-v1.c | ||
| dm-bio-prison-v1.h | ||
| dm-bio-prison-v2.c | ||
| dm-bio-prison-v2.h | ||
| dm-bio-record.h | ||
| dm-bufio.c | ||
| dm-builtin.c | ||
| dm-cache-background-tracker.c | ||
| dm-cache-background-tracker.h | ||
| dm-cache-block-types.h | ||
| dm-cache-metadata.c | ||
| dm-cache-metadata.h | ||
| dm-cache-policy-internal.h | ||
| dm-cache-policy-smq.c | ||
| dm-cache-policy.c | ||
| dm-cache-policy.h | ||
| dm-cache-target.c | ||
| dm-clone-metadata.c | ||
| dm-clone-metadata.h | ||
| dm-clone-target.c | ||
| dm-core.h | ||
| dm-crypt.c | ||
| dm-delay.c | ||
| dm-dust.c | ||
| dm-ebs-target.c | ||
| dm-era-target.c | ||
| dm-exception-store.c | ||
| dm-exception-store.h | ||
| dm-flakey.c | ||
| dm-historical-service-time.c | ||
| dm-init.c | ||
| dm-integrity.c | ||
| dm-io.c | ||
| dm-ioctl.c | ||
| dm-kcopyd.c | ||
| dm-linear.c | ||
| dm-log-userspace-base.c | ||
| dm-log-userspace-transfer.c | ||
| dm-log-userspace-transfer.h | ||
| dm-log-writes.c | ||
| dm-log.c | ||
| dm-mpath.c | ||
| dm-mpath.h | ||
| dm-path-selector.c | ||
| dm-path-selector.h | ||
| dm-queue-length.c | ||
| dm-raid.c | ||
| dm-raid1.c | ||
| dm-region-hash.c | ||
| dm-round-robin.c | ||
| dm-rq.c | ||
| dm-rq.h | ||
| dm-service-time.c | ||
| dm-snap-persistent.c | ||
| dm-snap-transient.c | ||
| dm-snap.c | ||
| dm-stats.c | ||
| dm-stats.h | ||
| dm-stripe.c | ||
| dm-switch.c | ||
| dm-sysfs.c | ||
| dm-table.c | ||
| dm-target.c | ||
| dm-thin-metadata.c | ||
| dm-thin-metadata.h | ||
| dm-thin.c | ||
| dm-uevent.c | ||
| dm-uevent.h | ||
| dm-unstripe.c | ||
| dm-verity-fec.c | ||
| dm-verity-fec.h | ||
| dm-verity-target.c | ||
| dm-verity-verify-sig.c | ||
| dm-verity-verify-sig.h | ||
| dm-verity.h | ||
| dm-writecache.c | ||
| dm-zero.c | ||
| dm-zoned-metadata.c | ||
| dm-zoned-reclaim.c | ||
| dm-zoned-target.c | ||
| dm-zoned.h | ||
| dm.c | ||
| dm.h | ||
| Kconfig | ||
| Makefile | ||
| md-bitmap.c | ||
| md-bitmap.h | ||
| md-cluster.c | ||
| md-cluster.h | ||
| md-faulty.c | ||
| md-linear.c | ||
| md-linear.h | ||
| md-multipath.c | ||
| md-multipath.h | ||
| md.c | ||
| md.h | ||
| raid0.c | ||
| raid0.h | ||
| raid1-10.c | ||
| raid1.c | ||
| raid1.h | ||
| raid5-cache.c | ||
| raid5-log.h | ||
| raid5-ppl.c | ||
| raid5.c | ||
| raid5.h | ||
| raid10.c | ||
| raid10.h | ||