[{"data":1,"prerenderedAt":521},["ShallowReactive",2],{"nav-stories":3,"project-lleven-v2":61,"ref-\u002Fprojects\u002Flleven-v1":148},[4,16,25,34,43,52],{"id":5,"color":6,"extension":7,"image":8,"label":9,"link":10,"meta":11,"order":12,"stem":13,"text":14,"__hash__":15},"stories\u002Fstories\u002F01-data-center.yml",null,"yml","https:\u002F\u002Fimages.unsplash.com\u002Fphoto-1558494949-ef010cbdcc31?w=1080","DATA_CENTER","https:\u002F\u002Fx.com\u002Fabbeytetteh_",{},1,"stories\u002F01-data-center","Racking new servers. 40gbit backbone online.","0QUZQbaANhdO8WemZxkDdO7vbVopfnynHtH9FxBZb_w",{"id":17,"color":6,"extension":7,"image":18,"label":19,"link":6,"meta":20,"order":21,"stem":22,"text":23,"__hash__":24},"stories\u002Fstories\u002F02-thoughts.yml","https:\u002F\u002Fimages.unsplash.com\u002Fphoto-1498050108023-c5249f4df085?w=1080","THOUGHTS",{},2,"stories\u002F02-thoughts","Late night bug hunting. Found the memory leak.","Gd1am954aasY6HRHD7hCtOuessXb6zYZ8iizS501ICg",{"id":26,"color":27,"extension":7,"image":6,"label":28,"link":6,"meta":29,"order":30,"stem":31,"text":32,"__hash__":33},"stories\u002Fstories\u002F03-coding.yml","#3b82f6","CODING",{},3,"stories\u002F03-coding","Just thinking about how much easier life is with Swarm. https:\u002F\u002Fgoogle.com","-WTk-47jnLM-TZRWBg0VbJyZJfIM7FpQ5HGbc8LEdhQ",{"id":35,"color":6,"extension":7,"image":36,"label":37,"link":6,"meta":38,"order":39,"stem":40,"text":41,"__hash__":42},"stories\u002Fstories\u002F04-update.yml","https:\u002F\u002Fimages.unsplash.com\u002Fphoto-1591799264318-7e6ef8ddb7ea?w=1080","UPDATE",{},4,"stories\u002F04-update","New cluster nodes arrived. Prepping for installation.","kyT60N5C6Re_jMonZbgNy0PbQhzXmUWxDbD0D_v43ts",{"id":44,"color":45,"extension":7,"image":6,"label":46,"link":6,"meta":47,"order":48,"stem":49,"text":50,"__hash__":51},"stories\u002Fstories\u002F05-setup.yml","#86868b","SETUP",{},5,"stories\u002F05-setup","Optimizing the telemetry pipeline for 1M req\u002Fs.","cPOBkzoyXsCmPgRO2d80Hj3vm4MP-6nAejtlQ5iuSzw",{"id":53,"color":6,"extension":7,"image":54,"label":55,"link":6,"meta":56,"order":57,"stem":58,"text":59,"__hash__":60},"stories\u002Fstories\u002F06-travel.yml","https:\u002F\u002Fimages.unsplash.com\u002Fphoto-1560969184-10fe8719e047?w=1080","TRAVEL",{},6,"stories\u002F06-travel","Travel log — system architecture workshop in Berlin.","jnOxerdF6usAIHdR35Z-opx0LJAy9kZluXnZhtz62Z0",{"id":62,"title":63,"body":64,"description":135,"extension":136,"liveUrl":137,"meta":138,"navigation":139,"order":12,"path":140,"seo":141,"stack":142,"stem":146,"thumbnail":6,"__hash__":147},"projects\u002Fprojects\u002Flleven-v2.md","Lleven V2: Scale & Security",{"type":65,"value":66,"toc":129},"minimark",[67,71,75,80,83,106,109,113,116,119,123,126],[68,69,70],"p",{},"Following the success and the hard lessons learned from the initial prototype, Lleven V2 was completely re-architected. Where V1 hit memory limits and HTTP timeouts with large PDFs, V2 adopts a robust, event-driven architecture designed for scale and security.",[72,73],"reference",{"path":74},"\u002Fprojects\u002Flleven-v1",[76,77,79],"h2",{"id":78},"the-asynchronous-pipeline","The Asynchronous Pipeline",[68,81,82],{},"To solve the \"Timeout Wall\" of V1, the entire parsing engine was decoupled from the HTTP request cycle.",[84,85,86,94,100],"ol",{},[87,88,89,93],"li",{},[90,91,92],"strong",{},"Upload:"," Files are securely uploaded and placed into an encrypted object store.",[87,95,96,99],{},[90,97,98],{},"Queueing:"," An event is fired to Redis Streams, queuing the document for processing.",[87,101,102,105],{},[90,103,104],{},"Background Workers:"," Dedicated consumer services written in Go pick up the job, run the intensive parsing, and update the database asynchronously.",[68,107,108],{},"This means a user uploading a 200-page statement gets an immediate response, while the system processes the file safely in the background.",[76,110,112],{"id":111},"identity-and-persistence","Identity and Persistence",[68,114,115],{},"A core limitation of V1 was the lack of user identity. In V2, we introduced a robust identity management system. Users can securely authenticate, manage their accounts, and review their processing history.",[68,117,118],{},"Data is no longer strictly ephemeral in a volatile cache; insights are persisted securely, allowing users to track their financial trends over time without re-uploading the same documents.",[76,120,122],{"id":121},"real-time-telemetry","Real-time Telemetry",[68,124,125],{},"With the new architecture, we implemented Server-Sent Events (SSE) to push real-time status updates back to the UI. The frontend (rebuilt in Nuxt and Vue) provides a live, reactive experience as the statement goes through validation, extraction, and analytics stages.",[68,127,128],{},"V2 sets a new standard for performance, reliability, and user experience.",{"title":130,"searchDepth":21,"depth":21,"links":131},"",[132,133,134],{"id":78,"depth":21,"text":79},{"id":111,"depth":21,"text":112},{"id":121,"depth":21,"text":122},"The next generation of the Lleven engine, featuring asynchronous processing, resilient data pipelines, and a secure user identity layer.","md","https:\u002F\u002Flleven.app",{},true,"\u002Fprojects\u002Flleven-v2",{"title":63,"description":135},[143,144,145],"Vue","Nuxt","Go","projects\u002Flleven-v2","OqhBSYnbfS0zohtEx0Nj449VSmaWGjiut8rKc8oODdA",{"id":149,"title":150,"body":151,"description":512,"extension":136,"liveUrl":6,"meta":513,"navigation":139,"order":21,"path":74,"seo":514,"stack":515,"stem":519,"thumbnail":6,"__hash__":520},"projects\u002Fprojects\u002Flleven-v1.md","Lleven V1: The Genesis",{"type":65,"value":152,"toc":498},[153,156,160,167,172,199,203,211,215,218,240,244,259,304,308,332,336,394,402,405,465,469,495],[68,154,155],{},"The first version of Lleven was born out of a simple need: to make sense of Mobile Money (MoMo) statements without the manual hassle. It was a lean, focused tool designed to transform a raw PDF into a \"Wrapped\" experience—much like Spotify, but for your spending.",[76,157,159],{"id":158},"architecture-overview","Architecture Overview",[68,161,162,163,166],{},"The V1 architecture was a classic ",[90,164,165],{},"Synchronous Processing"," model. It prioritized simplicity and immediate feedback for small-to-medium statements.",[168,169,171],"h3",{"id":170},"tech-stack","Tech Stack",[173,174,175,181,187,193],"ul",{},[87,176,177,180],{},[90,178,179],{},"Framework:"," FastAPI",[87,182,183,186],{},[90,184,185],{},"Processing:"," Pandas & PDFPlumber",[87,188,189,192],{},[90,190,191],{},"Caching:"," Redis",[87,194,195,198],{},[90,196,197],{},"Security:"," Fernet (AES-128) Encryption for cached data",[76,200,202],{"id":201},"the-processing-engine","The Processing Engine",[68,204,205,206,210],{},"Lleven V1 used a specialized parsing engine built on top of ",[207,208,209],"code",{},"pdfplumber",".",[168,212,214],{"id":213},"_1-validation-logic","1. Validation Logic",[68,216,217],{},"Before processing, the system checked for specific \"magic strings\" in the first page of the PDF to ensure it was a valid MTN MoMo statement. These included:",[173,219,220,225,230,235],{},[87,221,222],{},[207,223,224],{},"MSISDN:",[87,226,227],{},[207,228,229],{},"Time Run:",[87,231,232],{},[207,233,234],{},"TRANSACTION DATE",[87,236,237],{},[207,238,239],{},"ACCOUNT HOLDER NAME:",[168,241,243],{"id":242},"_2-data-extraction","2. Data Extraction",[68,245,246,247,250,251,254,255,258],{},"The engine targeted tables with a ",[207,248,249],{},"vertical_strategy"," and ",[207,252,253],{},"horizontal_strategy"," set to ",[207,256,257],{},"\"lines\"",". It mapped raw PDF columns to a structured internal format:",[173,260,261,294],{},[87,262,263,266,267,269,270,269,273,269,276,269,279,269,282,269,285,269,288,269,291,210],{},[90,264,265],{},"Raw Mapping:"," ",[207,268,234],{},", ",[207,271,272],{},"FROM ACCT",[207,274,275],{},"FROM NO.",[207,277,278],{},"TRANS. TYPE",[207,280,281],{},"AMOUNT",[207,283,284],{},"TO NO.",[207,286,287],{},"TO NAME",[207,289,290],{},"REF",[207,292,293],{},"OVA",[87,295,296,299,300,303],{},[90,297,298],{},"Cleaning:"," It used regex to identify the ",[207,301,302],{},"ACCOUNT_HOLDER_NO"," from the header text to distinguish between incoming and outgoing funds.",[168,305,307],{"id":306},"_3-data-cleaning-pipeline","3. Data Cleaning Pipeline",[173,309,310,320,326],{},[87,311,312,315,316,319],{},[90,313,314],{},"Date Normalization:"," Converted string dates (e.g., ",[207,317,318],{},"21-May-2023 10:30:00 AM",") into proper Python datetime objects.",[87,321,322,325],{},[90,323,324],{},"Type Casting:"," Converted currency strings into floats for arithmetic operations.",[87,327,328,331],{},[90,329,330],{},"Normalization:"," Removed newline characters and extra whitespace from names and references using custom regex cleaning.",[76,333,335],{"id":334},"the-upload-workflow","The Upload Workflow",[84,337,338,347,357,363,385],{},[87,339,340,343,344,210],{},[90,341,342],{},"Request:"," User uploads a PDF to ",[207,345,346],{},"\u002Fprocess-file",[87,348,349,352,353,356],{},[90,350,351],{},"Deduplication:"," A SHA-256 hash of the file is generated. If the hash exists in Redis, the system returns the existing ",[207,354,355],{},"file_hash"," immediately.",[87,358,359,362],{},[90,360,361],{},"Parsing:"," If new, the system extracts the table, cleans the data, and caps it to the requested year (e.g., 2023).",[87,364,365,368],{},[90,366,367],{},"Encrypted Caching:",[173,369,370,376,382],{},[87,371,372,373,210],{},"The resulting DataFrame is serialized using ",[207,374,375],{},"pickle",[87,377,378,379,210],{},"It is then encrypted using ",[90,380,381],{},"Fernet (symmetric encryption)",[87,383,384],{},"The encrypted blob is stored in Redis with a 1-hour TTL.",[87,386,387,390,391,393],{},[90,388,389],{},"Response:"," Returns the ",[207,392,355],{}," and an expiry timestamp.",[76,395,397,398,401],{"id":396},"the-retrieval-workflow-get-wrapped","The Retrieval Workflow (",[207,399,400],{},"\u002Fget-wrapped",")",[68,403,404],{},"When the user requests their \"Wrapped\" results:",[84,406,407,410,413],{},[87,408,409],{},"The system pulls the encrypted blob from Redis.",[87,411,412],{},"It decrypts and deserializes the DataFrame.",[87,414,415,418,419],{},[90,416,417],{},"On-the-Fly Analytics:"," It runs a series of summary algorithms:\n",[173,420,421,440,446,452],{},[87,422,423,426,427,269,430,269,433,436,437,210],{},[90,424,425],{},"Spending Summary:"," Aggregates totals for ",[207,428,429],{},"PAYMENT",[207,431,432],{},"CASH_OUT",[207,434,435],{},"TRANSFER",", and ",[207,438,439],{},"DEBIT",[87,441,442,445],{},[90,443,444],{},"Frequency Analysis:"," Calculates the top 5 recipients by amount and frequency.",[87,447,448,451],{},[90,449,450],{},"Monthly Trends:"," Groups transactions by month to visualize spending patterns.",[87,453,454,457,458,460,461,464],{},[90,455,456],{},"Credit Summary:"," Identifies salary or incoming transfers by filtering for the user's ",[207,459,302],{}," in the ",[207,462,463],{},"TO_NO"," column.",[76,466,468],{"id":467},"limitations-of-v1","Limitations of V1",[173,470,471,477,483,489],{},[87,472,473,476],{},[90,474,475],{},"The \"Timeout\" Wall:"," Large PDFs (50+ pages) often caused HTTP timeouts because the API waited for the entire extraction to finish before responding.",[87,478,479,482],{},[90,480,481],{},"Memory Pressure:"," Since processing happened on the API workers, high concurrent uploads could lead to OOM (Out of Memory) errors.",[87,484,485,488],{},[90,486,487],{},"Stateless Persistence:"," Data only lived in Redis. If the cache expired, the user had to re-upload the file.",[87,490,491,494],{},[90,492,493],{},"Lack of Identity:"," No user accounts meant users couldn't see a history of their past uploads without keeping the file hashes themselves.",[68,496,497],{},"Lleven V1 proved the concept, but the stage was set for a more robust, scalable, and secure V2.",{"title":130,"searchDepth":21,"depth":21,"links":499},[500,503,508,509,511],{"id":158,"depth":21,"text":159,"children":501},[502],{"id":170,"depth":30,"text":171},{"id":201,"depth":21,"text":202,"children":504},[505,506,507],{"id":213,"depth":30,"text":214},{"id":242,"depth":30,"text":243},{"id":306,"depth":30,"text":307},{"id":334,"depth":21,"text":335},{"id":396,"depth":21,"text":510},"The Retrieval Workflow (\u002Fget-wrapped)",{"id":467,"depth":21,"text":468},"The first version of Lleven, a synchronous processing engine designed to transform raw Mobile Money PDFs into structured spending analytics.",{},{"title":150,"description":512},[516,517,518],"FastAPI","Pandas","Redis","projects\u002Flleven-v1","HUCFXtkqCH7TKZwiJKSCxzJevh-30bUSAoJtS1E_9Mc",1780657374460]