Concurrent File Deduplicator using Ruby Threads
Build a command-line tool that scans a directory tree for duplicate files by content hash, using concurrent Ruby threads to hash files in parallel for speed on large directories. Learners practice Ruby concurrency primitives, file I/O, and hashing algorithms.
- Estimate
- ~9h
- Steps
- 5
- Completed by
- 0
- Proposed by
- codeseed.app
Digest
Project roadmap
- 01
Walk the directory tree
~1.5hRecursively enumerate all files in a target directory, collecting their paths and sizes.
- 02
Hash files concurrently
~2.5hUse a thread pool to compute SHA256 hashes of files in parallel, grouping results by hash.
- 03
Identify and report duplicates
~1.5hGroup files by identical hash and print groups containing more than one file as duplicates.
- 04
Add a dry-run and delete mode
~2hAdd a mode that lists space that could be reclaimed and an opt-in mode to delete all but one copy per duplicate group.
- 05
Benchmark concurrency gains
~1.5hCompare runtime between single-threaded and multi-threaded hashing on a large test directory and report the speedup.
Resources
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack