undefined | Better HN

0 pointsHarHarVeryFunny3y ago0 comments

So what is content of those various files? Does this include the full models themselves, or just the weights ?

0 comments

3 comments · 1 top-level

WithinReason3y ago· 2 in thread

The pth file seems to be a model and weights, saved as described here:

https://pytorch.org/tutorials/beginner/saving_loading_models...

.chk file is am md5 hash of the file, the .json file contains this for the 7B model:

    {"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-06, "vocab_size": -1}

HarHarVeryFunnyOP3y ago

Thanks, so from that PyTorch doc it seems that pickle format has the filenames of the model classes, but not the classes themselves. I'm sure someone will figure it out though!

WithinReason3y ago

The code (including the model) is here:

https://github.com/facebookresearch/llama

I already got the 7B model to generate text using my GPU! The 1st example prompt generated this:

[I believe the meaning of life is] to be happy, and it is also to live in the moment. I think that is the most important thing. I'm not really a party girl. I'm not a girl's girl. I have a really small group of close girlfriends and that's all I need. I believe in equal rights for everyone. I'm not a rebel. I don't really rebel against anything. I'm a very traditional girl, very loyal. I'm a mum's girl and I'm a dad's girl. People have a right to know what's going on. I don't care about the haters, because at the end of the day they're just going to have to deal with themselves. I've been getting more and more into fashion since I was about 16. I know I'm a little different, but so what? I think that's good. I don't think you should be like everyone else. It's my birthday, and I'll cry if I want to. I've always been a huge fan of fashion, and I've always liked to dress up

1 more reply

j / k navigate · click thread line to collapse