Redis Data Cleaner
Build a Lua tool that checks the data stored in a Redis database and finds broken records. You describe what each group of keys should look like (for example, user:* keys are hashes with a valid email and a numeric age, and session:* keys hold JSON with an expires_at field), and the tool scans the matching keys and reports every value that doesn't follow the rules. It never blocks Redis: it uses SCAN instead of KEYS. It only reports by default, and can optionally move bad records to a quarantine prefix. It runs with OpenResty's resty command, using lua-resty-redis and the built-in cjson library. You'll learn Redis data types, safe key scanning, writing validation rules in Lua, and designing a tool that's careful with production data.
- Estimate
- ~12h
- Steps
- 7
- Completed by
- 0
- Proposed by
- codeseed.app
OpenResty · lua-resty-redis · Redis · LuaRocks · busted
Project roadmap
- 01
Set up Redis and sample data
~1.5hInstall OpenResty (which includes the resty command and lua-resty-redis) and run Redis locally, for example with Docker. Write seed.lua that fills Redis with about 1,000 sample keys (user:* hashes and session:* JSON strings), with about 5% broken on purpose: bad emails, missing fields, invalid JSON, wrong types. Done when: resty seed.lua fills Redis, and redis-cli DBSIZE shows the keys.
- 02
Describe the rules
~1.5hCreate rules.lua returning a table of rules. Each rule has a key pattern (such as user:*), the expected Redis type (hash, string, list or set), and checks: required fields, which fields must be numbers, which must look like an email (use a Lua pattern), and whether a string value must be valid JSON with certain fields. Done when: rules.lua describes the users and sessions in the sample data.
- 03
Scan keys safely
~2hConnect with lua-resty-redis and set a timeout. For each rule, loop with SCAN using MATCH with the rule's pattern and COUNT 100 until the cursor returns to 0, so Redis is never blocked the way KEYS would block it. Never use KEYS. Add a --limit option to stop after N keys. Done when: the tool lists how many keys match each rule.
- 04
Validate values
~2.5hFor each key, check its type with TYPE, then read the value with the matching command (HGETALL for hashes, GET for strings) and apply the rule's checks. JSON is parsed with cjson.safe so bad JSON returns an error instead of crashing. For each broken key, record the key and every reason it failed. Done when: the tool finds the records you broke on purpose in seed.lua, and no valid ones.
- 05
Write the report
~1.5hPrint a summary per rule (keys scanned, valid, broken) and a list of broken keys with their reasons. Add --output report.json to save the full report as JSON. Exit with code 1 if broken records were found. Done when: the summary numbers match the broken records you created.
- 06
Quarantine broken records
~1.5hAdd a --quarantine option that, only when also given --yes, renames each broken key to quarantine:<original key> with RENAMENX, so nothing is overwritten, and logs every rename. Without --yes, print what would be renamed. Done when: running with --quarantine --yes moves exactly the broken keys, and running the check again reports none.
- 07
Test the validation rules
~1.5hSeparate the pure validation functions (the checks that take a value and return errors) from the Redis code. Test them with busted, installed through LuaRocks, using good and bad example values: valid and invalid emails, missing fields, broken JSON and non-numeric numbers. Done when: busted passes.
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack