I run Mistral-7B on an old laptop. It's not very fast and it's not very good, but it's just good enough to be useful.
My use case is that I'm more productive working with a LLM but being online is a constant temptation and distraction.
Most of the time I'll reach for offline docs to verify. So the LLM just points me in the right direction.
I also miss Google offline, so I'm working on a search engine. I thought I could skip crawling by just downloading common crawl, but unfortnately it's enormous and mostly junk or unsuitable for my needs. So my next project is how to data-mine common crawl to extract just the interesting (to me) bits...
When I have a search engine and a LLM I'll be able to run my own Phind, which will be really cool.
Presumably you could run things like PageRank, I'm sure people do this sort of thing with CommonCrawl. There are lots of variants of graph connectivity scoring methods and classifiers. What a time to be alive eh?
My use case is that I'm more productive working with a LLM but being online is a constant temptation and distraction.
Most of the time I'll reach for offline docs to verify. So the LLM just points me in the right direction.
I also miss Google offline, so I'm working on a search engine. I thought I could skip crawling by just downloading common crawl, but unfortnately it's enormous and mostly junk or unsuitable for my needs. So my next project is how to data-mine common crawl to extract just the interesting (to me) bits...
When I have a search engine and a LLM I'll be able to run my own Phind, which will be really cool.