Merry Christmas!
Was watching one of George Hotz's livestreams where he was working on tinygrad library. I was always confused about what he was doing, but then I saw this statement: "tinygrad will always be below 1000 lines. If it isn't, we will revert commits until tinygrad becomes smaller." So beautiful. Everything made sense. Tinygrad was just a bigger version of karpathy's micrograd . Anyway, this is what I ended up reading only to realize that while I was explicitely defining the derivatives in raML, micrograd seems to refer to sympy for gradients. While I was aware that sympy could do it, I found it to be slow in the past, but maybe it isn't as bad as I thought. Gonna try out. (PyTorch?) Also tinygrad defines a Tensor class which generalizes numpy's ndarray, which is also interesting. Another idea: expose model weights via some API. FastAPI maybe? Another idea 2: I should DarkMode this website