Homemade GPT JS – A Tensorflow.js Re-Implementation of MinGPT (opens in new tab)

(github.com)

2 pointstrekhleb1y ago2 comments

2 comments

That's a fantastic project! Thank you for enabling training the model directly in the browser!

I have previously been impressed by https://github.com/0hq/WebGPT. Now, your project has rekindled hopes for a pure-JS GPT-like model.

Keep it up!

trekhlebOP1y ago

Thanks for the feedback! WebGPT is good. Looks like it is a vanilla JS? I used TensorFlow.js to offload all the troubles of working with tensors, gradients, and WebGPU integration to it. Along with a possibility to train the model in the browser it also helped to keep the actual GPT code pretty concise (<300 lines). Hopefully it will make easier to learn the model architecture itself for those who’re interested.

j / k navigate · click thread line to collapse

2 comments

felladrin1y ago

That's a fantastic project! Thank you for enabling training the model directly in the browser!

I have previously been impressed by https://github.com/0hq/WebGPT. Now, your project has rekindled hopes for a pure-JS GPT-like model.

Keep it up!

trekhlebOP1y ago

j / k navigate · click thread line to collapse