Hi everyone, so this week has been comparatively more productive in terms of learning as well as work! I’m happy, so why not share with everyone as well? 🎉

PyTorch: (ft. Quansight and Facebook)

  • Started working on porting index_add to structured kernels, see the PR I made on my forked repo here, and on upstream here.
    • This included adding an out= variant to the op.
    • Refining the way it’s registered in PyTorch. I am thinking to pass defalut value while registering, but it will be a BC breaking change.
    • Finally got to use ghstack, thanks to Yukio (Quansight).
    • Revised derivatives yaml file. Personally I feel that this one needs opinions from the Facebook team, and a lot of changes might be rejected (which is okay, at the end of the day - everything we do should be good for the library).
    • My PR here is more like a prototype for everyone to get a chance to review, as well as comment on what they feel.
  • Structured Kernel porting PR for baddbmm, bmm has been merged. Yay! 🎉 ❤️ PR.
  • Took a walkthrough of lots of autogenerated code in PyTorch, to understand how ops are registered.
  • Listened to Ed’s podcast - episode on NVIDIA GPUs. A sweet little introduction to NVIDIA GPUs.

Extras:

  • Continued reading history of JIT, but couldn’t complete it. :sad: Will have to continue reading the next week, find the link here.
  • Effective Modern C++: I finished reading Item 14 of Chapter 3, but couldn’t write notes about it. This blog post, I’m planning to discuss the intermediate generated code (closer to assembly), so let’s see how that goes the next week.
  • Learnt a bit about compilers, started reading a book on it.
  • Wrote a shell script in case I distro hop a lot, will link on this later once I’ve uploaded it.
  • Started going through NumPy’s source code, just something I’m doing for learning.
  • Reading about memory overlaps in C.
  • BuffetCodes organization was dissolved recently, and the repositories were transferred. Thanks to my friend (Himanshu Singh) on working with me on this, it was a great experience.

I’m continuously thinking if writing this is anyway helpful for anyone else, so yes - please do let me know if you like reading these. However, I’m enjoying learning and I think that’s all that matters. I plan to start new projects very soon, and hopefully that should go good.