About Differentiable Programming

The Discovery of an Interesting Book: The Elements of Differentiable Programming

Recently, while scrolling through LinkedIn, I came across a post about the book The Elements of Differentiable Programming, written and shared for free by DeepMind researchers.

What caught my eye was that Yann LeCun, one of my favourite computer scientists, had pressed the “Recommend” button on this post. As a follower, it appeared on my feed with the line: “If you want to write code that learns, this book is for you.”

(Something like this, I can’t remember exactly)

I couldn’t help but start researching it right away.

Why This Resonates With Me

The reason I chose computer science as my major is rooted in my curiosity about artificial intelligence and its foundations.

So far throughout my uni journey, I’ve learned so many different things:

• Programming languages: C, Java, Python…
• Object-Oriented Programming and Software Engineering
• CS theory, computer communications, and networking
• Mobile & web development, databases, frontend design in Figma
• Collaborative software projects following Agile, DevOps practices, containerization, etc
• machine learning

Now, with just one semester left (July 2025), I can honestly say I’ve enjoyed every part of my degree. But I still feel like I want to go deeper into the areas that truly interest me especially AI.

I’ve often thought about pursuing a Master’s or PhD to get there someday, but at this point, I’m more focused on gaining real-world experience through an internship or graduate role.

During the capstone project, I really enjoyed the moment that I, as a software developer, could help clients (building a web app for their logistics work) and felt fulfilled.

So I am down for anything if I can be part of a meaningful project, even if it is not AI-related, and I love software development too.

Expanding My AI Knowledge

However, I believe it is always better to learn something new. I’ve been actively seeking out extra AI resources and learning opportunities. I’ve:

• Attended workshops and seminars
• Completed Microsoft AI Skills for Students courses
• Studied for the AWS AI Practitioner certification (which broadened my perspective, though I realised it’s quite entry-level when looking at the AWS certification roadmap)

So finding this book now feels like perfect timing, thanks LinkedIn!😆 With a few weeks left before my second semester starts, I plan to dive in.

First Impressions of the Book

I could grab the PDF version of the book here: arXiv.org link

As of now, it’s version 3, last updated 24 June 2025.

Before starting, I was a bit nervous because it was my first time seeing terms like Differentiable Programming and autodiff. But fortunately, the explanations have been understandable so far (maybe it’s because only the introduction and the first chapter).

The book explores the core concepts behind how machine learning models, especially deep neural networks, are trained using gradient-based optimisation.
Because I’d completed a Machine Learning unit in Semester 1 and also watched Andrew Ng’s Coursera ML specialist courses, these ideas felt familiar. But in practice, what I actually did in labs and projects relied heavily on well-known frameworks like PyTorch and TensorFlow.

So this book feels like a perfect chance to peel back the layers and understand the mathematics that power those frameworks.

Treating It Like an Extra Unit

From my uni experience, one unit per semester typically aims to cover a textbook like this. For example, in Theoretical Computer Science (COMP3002), we covered Introduction to the Theory of Computation by Michael Sipser. That was tough, a few hundred pages written in formal language that required extra effort to fully digest.

Luckily, I found Michael Sipser’s MIT lectures on the MIT OpenCourseWare YouTube channel. They were a huge help when tackling that unit.

Ready for the Challenge

In a way, reading The Elements of Differentiable Programming will be like taking on an extra unit next semester. Even though it won’t be formally credited, I’m excited to challenge myself and learn how to write code that can truly learn for itself.

This is exactly the kind of deep dive I’ve been looking for, and I’m ready for it.😊