Micrograd - A tiny Autograd engine (with a bite! :))
packages = [
"micrograd",
"numpy",
"matplotlib"
]
Micrograd is
a tiny Autograd engine created by
Andrej Karpathy. This
app recreates the
demo
he prepared for this package using pyscript to train a basic
model, written in Python, natively in the browser.
You may run each Python REPL cell interactively by pressing
(Shift + Enter) or (Ctrl + Enter). You can also modify the code
directly as you wish. If you want to run all the code at once,
not each cell individually, you may instead click the 'Run All'
button. Training the model takes between 1-2 min if you decide
to 'Run All' at once. 'Run All' is your only option if you are
running this on a mobile device where you cannot press (Shift +
Enter). After the model is trained, a plot image should be
displayed depicting the model's ability to classify the data.
Currently the > symbol is being imported
incorrectly as > into the REPL's. In this
app the > symbol has been replaced with
().__gt__() so you can run the code without issue.
Ex: instead of a > b, you will see
(a).__gt__(b) instead.
import js; js.document.getElementById('python-status').innerHTML = 'Python is now ready. You may proceed.'
Python is currently starting. Please wait...
import random import numpy as np import matplotlib.pyplot as plt
from micrograd.engine import Value from micrograd.nn import Neuron,
Layer, MLP
np.random.seed(1337) random.seed(1337)
#An adaptation of sklearn's make_moons function
https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html
def make_moons(n_samples=100, noise=None): n_samples_out,
n_samples_in = n_samples, n_samples outer_circ_x =
np.cos(np.linspace(0, np.pi, n_samples_out)) outer_circ_y =
np.sin(np.linspace(0, np.pi, n_samples_out)) inner_circ_x = 1 -
np.cos(np.linspace(0, np.pi, n_samples_in)) inner_circ_y = 1 -
np.sin(np.linspace(0, np.pi, n_samples_in)) - 0.5 X =
np.vstack([np.append(outer_circ_x, inner_circ_x),
np.append(outer_circ_y, inner_circ_y)]).T y =
np.hstack([np.zeros(n_samples_out, dtype=np.intp),
np.ones(n_samples_in, dtype=np.intp)]) if noise is not None: X +=
np.random.normal(loc=0.0, scale=noise, size=X.shape) return X, y X,
y = make_moons(n_samples=100, noise=0.1)
y = y*2 - 1 # make y be -1 or 1 # visualize in 2D
plt.figure(figsize=(5,5)) plt.scatter(X[:,0], X[:,1], c=y, s=20,
cmap='jet') plt
model = MLP(2, [16, 16, 1]) # 2-layer neural network print(model)
print("number of parameters", len(model.parameters()))
Line 24 has been changed from: accuracy = [(yi > 0) == (scorei.data > 0) for yi, scorei
in zip(yb, scores)]
to: accuracy = [((yi).__gt__(0)) == ((scorei.data).__gt__(0)) for
yi, scorei in zip(yb, scores)]
# loss function def loss(batch_size=None): # inline DataLoader :) if
batch_size is None: Xb, yb = X, y else: ri =
np.random.permutation(X.shape[0])[:batch_size] Xb, yb = X[ri], y[ri]
inputs = [list(map(Value, xrow)) for xrow in Xb] # forward the model
to get scores scores = list(map(model, inputs)) # svm "max-margin"
loss losses = [(1 + -yi*scorei).relu() for yi, scorei in zip(yb,
scores)] data_loss = sum(losses) * (1.0 / len(losses)) # L2
regularization alpha = 1e-4 reg_loss = alpha * sum((p*p for p in
model.parameters())) total_loss = data_loss + reg_loss # also get
accuracy accuracy = [((yi).__gt__(0)) == ((scorei.data).__gt__(0))
for yi, scorei in zip(yb, scores)] return total_loss, sum(accuracy)
/ len(accuracy) total_loss, acc = loss() print(total_loss, acc)
# optimization for k in range(20): #was 100. Accuracy can be further
improved w/ more epochs (to 100%). # forward total_loss, acc =
loss() # backward model.zero_grad() total_loss.backward() # update
(sgd) learning_rate = 1.0 - 0.9*k/100 for p in model.parameters():
p.data -= learning_rate * p.grad if k % 1 == 0: print(f"step {k}
loss {total_loss.data}, accuracy {acc*100}%")
Please wait for the training loop above to complete. It will not
print out stats until it has completely finished. This typically
takes 1-2 min.
Line 9 has been changed from: Z = np.array([s.data > 0 for s in scores])
to: Z = np.array([(s.data).__gt__(0) for s in scores])