Its presumably llama just like Mistral. *Everything* open source is llama now. F... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		brucethemoose2 on Dec 8, 2023 \| parent \| context \| favorite \| on: Mistral "Mixtral" 8x7B 32k model [magnet] Its presumably llama just like Mistral. Everything open source is llama now. Facebook all but standardized the architecture. I dunno about the moe. Is there existing transformers code for that part? It kinda looks like there is based on the config.

jasonjmcghee on Dec 8, 2023 | [–]

Mistral is not llama architecture.

https://github.com/mistralai/mistral-src

brucethemoose2 on Dec 8, 2023 | | [–]

Its basically llama architecture, all but drop in compatible with llama runtimes.

refulgentis on Dec 8, 2023 | [–]

Because it's JSON? :)

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact