Lovely day, had my friend imsrbh come over and we were talking about the project I wanted to do. He had some ideas and of course some experience with Kafka, so he helped me setup Kafka and docker on my machine, and that’s how it all started. Setting up Kafka and Docker on my machine: Refer to this blog for Kafka setup. We verified by running a sample producer-consumer app in Python and it worked. Starting to implement the core logic of getting key strokes in Rust I’m using rdev library for this, it was decent. Was wondering how I can convert the keystrokes to strings, apparently it was a Rust Enum. Fortunately, they had a feature serialize which would help me do just that. Now it’s quite smooth. Implementing Kafka Producer-Consumer in Rust Kafka has a client in Rust, cargo add kafka will help. Wrote Kafka Producer and Consumer referring to their official documentation. It works flawlessly. We were sending the keys as &[u8] and had to convert these to strings back at consumer (need to proper error checking there) Producer repo: https://github.com/krshrimali/keystroke-store-rs Consumer repo: https://github.com/krshrimali/keystroke-consumer-rs Quick learning on why we didn’t go for Redis (mostly because we wanted to learn Kafka in Rust lol, can try Redis one day but I like reliability of Kafka): https://stackoverflow.com/a/37993809 A lot of TODOs from yesterday are left, I’ll move them to tomorrow, yay! Procrastination, let’s go 😆🎉 ...

Alright, we are back. It’s Saturday, so I spent a lot of time cleaning up my flat and my setup. Bengaluru is quite a dusty city, so yep, gotta clean regularly. Anyways, in terms of learning, my goals of today were to read some source code and papers/official documentation. It’s been some time since I didn’t do any Rust, so today I’ll just touch upon that as well. Rust - building a UI to plot a sensor in real time Why this? I’ve used Iced library quite a bit so far, so wanted to understand how other UI libraries look like. I’m not going for the “famous” libraries here, just wanted to experience any other library. Plus, this video seemed to be decent and technically focused enough on the implementation. Setting up PostgreSQL server on my system: Why this? For me to start working on some projects of my own, it’s important to setup stage for them. Notes: I used this blog by digital ocean as reference. Micro Project: Store all the key presses in a server Notes: Since I type a lot, everyone does, the DB will have to be good enough. I wanted to see how I can enable indexing and searching, this will help give me an idea. No UI stuff right now. Maybe one day. NeoVim Config: I was randomly trying a keymap, and I just remembered I had <leader>nf for SnipRun. And oh, wow - it finally made sense why it could be useful. Selecting text, code block, displaying output in the command window or as virtual text (different highlight for errors) and even in a vertical split terminal, amazing stuff! Something my nvim-autorunner was trying to do, cool stuff. Oh, and on another note: I watched a movie (after a long long time). Had to find time to relax a bit. ...

Started my day with talking to my family, and then kicking off work after getting ready. (Note: these blogs are mostly around my personal learning updates, so I’ll mostly miss anything that I do at work as that’s confidential) What is OSI Model | Real World Examples Why this? The answer to: “How does the communication work from client to server” is very important in backend. I wanted to see if I missed anything from my graduate studies, but nothing much. This was more like a refresher. Notes: Each operating system has their own implementation of TCP protocol. Linux has it’s own for example, and it’s open sourced. Can take a look at TCP’s source code in Linux source code to understand how checksum correction works. Cloud load balancers: L4 Load Balancer (operates at TCP Level), L7 Load Balnacer (operates at Application Protocol Layer - HTTP/HTTPS) Postgres: Architectural Fundamentals Why this? Going through database design and architectural fundamentals is helpful. I plan to explore CockroachDB soon, but wanted to get some idea about the ones I’ve used so far (Postgres for example) Notes: Uses client/server model. PostgreSQL session consists following processes: Server: manages DB files, accepts connections to the DB from client apps, performs DB actions on behalf of clients. DB server process is called postgres Client (frontend): app that wants to perform DB operations. Client/server could be on diff hosts: communicate over TCP/IP n/w connection postgres can handle multiple connections from clients: Forks a new process for each connection The new process and client directly communicate No intervention of postgres server Supervisor server process is always running (daemon process), waiting for client connections Client and child server process can come and go PostgreSQL System Architecture Why this? Just wanted to explore another blog on the same topic to see if I missed anything. Tbh, nothing much. Notes: PostgreSQL: process-per-transaction model postgres server process: Managed by postmaster, central coordinating process Responsibilities: Initializing, terminating the server Handling connection requests from the new clients Recovery Run background processes Shared Memory: Reserved for DB caching and Transactional Log Caching Shared Disk Buffer Shared Tables Uses same set of tables to host multiple client data (TODO: how?) Backend Processes: Client interacts with backend processes (submits queries and receiving queries result) Multiple backend servers executing queries concurrently Each backend server: will handle only a single query at a time access data from main memory buffer pool (placed in shared memory) Thank you! :) See y’all in the next blog. ...

Hi everyone!! I’ve an update, on what I’ve been up to - and I’m excited. It’s a rusty update 😉. As always, I would love to tell you a story (this will help set some context), but if you want to skip and go directly to the update, please scroll to: # THE Update section. The Story I’ve been through, possibly, the best and the worst time of my life. I use these contrasting words because everything went wrong - and that’s where you get an opportunity to shine. I think I fairly enjoyed resisting the pain and fighting through the odds, hopefully you’ll learn about it very soon on my blog (just need the courage to say it out loud :)). ...

Hi everyone! Sorry for missing out on publishing individual blogs for 19th, 20th and 21st November, but it has been very hectic for me, and I decided to give leetcode a break and explore System Design (a lot). Turns out, I had some idea about it already, one of the advantages of working with startups (CareAI/Dukaan). But in any case, quite a few interesting things I learnt. I can’t share all the links, as it will be just better to share the playlists or YT channels I referred: ...

Hi everyone, Day 5 of this series of daily updates. Started my day, fairly early but was feeling unwell. In any case, I did make some progress with my learning of the Interpreter. One of motivation to learn about Interpreter is to learn the whole process, from the code we write to the object files / executables. Of course, in this goal, my next step would be to read the “Write a compiler in Go” book. But one step at a time, right? ...

Hi everyone, Day 4 of this series of daily updates. I don’t have a lot to share today, just one of those days where I spent a lot of time going through whatever I’ve done already. I take some time to look at the things I learned so far, and see if I could be more intuitive back then. I’m going to continue doing the same for the next day, though this time for the problems I’ve solved on Leetcode. ...

Hi everyone, Day 3 of this series of daily updates. Woke up fairly early (~6 AM), and started working. Though it seemed productive to me, but you won’t see a lot of content here today - just one of those days where I didn’t do a lot of different things, I guess. :) Projects/Learning: I made my rust-leetcode repository public today. It currently contains 12 solved problems, you’ll see huge difference between the first few codes to the last few codes. I’ve been learning, please don’t refer it for learning. I’m just documenting it there. Autogenerate README once a new file is added to my Rust Leetcode repository: I first started with writing a Python script to auto-generate the README.md file, from the modules registered in the src/lib.rs. The script is here. Since I wanted to explore how it can be done in Rust, I ported it to Rust as well. The script is here. Took some break from doing leetcode today, had a lot of work which I unfortunately can’t mention here. ...

Hi everyone, Day 2 of this series of daily updates. Started my day earlier than yesterday, at around 9 AM (work). Learning: Watched this nice video from Jason on The Right Way to Write C++ in 2022: I like when he advocated on using different compilers, or at least verifying that it works for all (and if not, then you’re aware). Interest: take warnings seriously. Need to start doing it, I’ve been following this with Rust, but C++ never raised Rust-like “good enough” warnings. Might as well start doing it now. Code coverage analysis: hmm, it used to stay at around 87 to 90% for the libraries I worked with. I agree, most of it should be tested, but as the library grows longer, you can’t block PRs on coverage (specially by the community). But it’s underrated, I’ve seen developers ignore it until they face the wrath of weird bugs. 100% testing coverage, will always be difficult, but if the goal is kept in that direction, it might reach to 95-100%. Never heard of fuzz testing before…after reading about it, just another word of what we have done before. Never got a name to it though. ;) Ship with Hardening Enabled: Again, no idea about this. I’ll have to read about it. Marked as a TODO. Missed watching Edward’s stream on TorchDynamo, I woke up late. :/ Going to re-watch the stream now. I’ll share the notes in a separate blog once I’m done, I could only watch some of it today. I’ve started learning Kernel Fusion, how does it work and why it can help (and when it can not). This is going to be a long road, hopefully I’ll be able to share something concrete soon. Leetcode Problems: ...

Hi everyone, I’m back to - writing daily update blogs. Just sharing what I did today, and documenting my progress. Now of course it won’t contain everything I do, but yep - everything related to learning and things I’m passionate about. Reading Just In Time Compilation (Wiki): Wiki: https://en.wikipedia.org/wiki/Just-in-time_compilation Honestly, I wanted to start with a few papers on it, but Wikipedia does give enough background and sets some context. Leetcode problems ...