TinyTorch, close to the metal

[sueszli]

Project: https://github.com/sueszli/autograd.c/

After working through the TinyTorch book, I reimplemented the core ideas and exercises in C to force a lower-level understanding of autograd and ML system mechanics.

The result is a small PyTorch-like framework built from scratch in C, including tensor ops and a reverse-mode autograd engine. The project was primarily an exercise in understanding design tradeoffs, memory layout (e.g. Contiguous Allocation, Memory Arenas) and execution at the systems level.

Sharing it here in case it is useful or interesting to others working through TinyTorch 🌸 ✨

Thanks to Prof. Vijay Janapa Reddi and the TinyTorch community for the material and inspiration!

Yahya

[profvjreddi]

Very cool @sueszli! Reimplementing autograd in C is a great way to really internalize what's going on at the systems level. Memory arenas, contiguous allocation, manual lifetime management — you're confronting decisions that higher-level frameworks hide. Thanks for sharing this. I have actually seen yours and it is nice! and also I do think I replied in the past 😓 but can't remember 😅

Anwyays as you know -- If you're interested, we also have TinyTorch at tinytorch.ai as the hands-on companion to the textbook, plus hardware kits at mlsysbook.ai/kits (Arduino, Raspberry Pi, Seeed). We're also working on interactive labs that should be out soon. Would love your feedback given your low-level systems perspective.

[sueszli]

Awesome, thanks! I was actually thinking more along the lines of a quick call if you're up for it -- happy to work around your schedule.

[profvjreddi]

I'm incredibly swamped right now getting ready for some major releases on our projects for the MLSys book, so how about we catch up on this in a couple of weeks?

[sueszli]

Completely understand, no rush at all. I would love to learn more about your work and how I might be able to contribute.

I would be happy to give you a 5min elevator pitch whenever suits you. I am based in Zurich, Switzerland but flexible on time and can match your time zone. Feel free to reach out at [lastname][firstname]@gmail.com, even a day ahead is plenty of notice.

[profvjreddi]

Are you at ETH by any chance? I am doing my sabbatical at ETH starting in July, so it might just be easier to meet in person 😅

[sueszli]

Yes, that would also work.

I'm looking forward to meeting you in person
and really hope you enjoy your time in Switzerland.