The entire series is at https://karpathy.ai/zero-to-hero.html, first video is literally from scratch (i.e. just Python, the only external dependency is for plotting graphs if I recall correctly).
Andrej's "Building Makemore" series is exactly this, it includes a wonderful lecture where he computes all the gradients for a simple network by hand and compares them against the values produced by torch's autograd.
Same, but I would honestly rather we get away from silicon or anything resembling what we call these "computers" these days. Things are too complicated and we need to truly get back to the basics: rocks on a hill.
