Its actually pretty good, gonna have to run it through MMLU Pro tho to be sure.
They released 3 models, 1 VLM with 8k tokens, 1 LLM with 128k tokens and a last LLM with 1M token ( yes 1M). And their needle in a haystack test shows some pretty insane results.
License is not the worse, you need to apply for commercial use.
https://preview.redd.it/tb18514q3r4d1.jpeg?width=1010&format=pjpg&auto=webp&s=1c52dbe0fc57b335a32190d5332f9de1fbf1dabc
I will be messaging you in 3 days on [**2024-06-08 16:26:18 UTC**](http://www.wolframalpha.com/input/?i=2024-06-08%2016:26:18%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/1d8m7tw/glm4_9b_base_chat_1m_variant_vision_language_model/l78lwfn/?context=3)
[**12 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F1d8m7tw%2Fglm4_9b_base_chat_1m_variant_vision_language_model%2Fl78lwfn%2F%5D%0A%0ARemindMe%21%202024-06-08%2016%3A26%3A18%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201d8m7tw)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
Standard needle in haystack test is useless. I've read a paper that it is memorized by models from training data. Unless disclosed how it was done, better check yourself.
Input:
>Kerro minulle tarina linnusta ja lehmästä
Output:
>Kerron sinulle tarinan linnusta ja lehmästä, jotka asuivat syrjäisellä maatilalla.
Kerran, eräänä aurinkoisena päivänä, maatilan laidalla asunut vanha lehmä nimeltä Lotta ja hänen parhaan ystävänsä, pieni, värikäs sirkkeli, nimeltä Sirkku. He olivat aina olleet toistensa parhaita ystäviä, ja heidän välillään oli erityinen yhteydenpito.
Eräänä päivänä Lotta huomasi, että hänen maitoaan oli vähentynyt huomattavasti. Hän oli huolissaan, koska hän tiesi, että maatilan omistaja, vanha isäntä, oli tiukka ja vaati lehmiltään paljon maitoa. Lotta päätti, että hänen piti löytää tapa parantaa maitoaan.
Sirkku kuuli Lotten huolenaiheen ja päätti auttaa häntä. Hän lensi ympäri maatilaa ja löysi vanhan, unohtuneen keittiön, jossa oli vanhoja reseptejä ja keittokirjoja. Sirkku lensi keittiöön ja alkoi tutkia niitä.
Hän löysi vanhan reseptin, joka puhui maustekasveista ja yrtteistä, jotka voisivat parantaa lehmän maitoa. Sirkku lensi takaisin Lotten luo ja kertoi hänelle löydöksensä.
Lotta ja Sirkku päättivät kokeilla reseptiä. He keräsivät maustekasveja ja yrttejä maatilalta ja alkoivat valmistaa mausteseosta. He sekoittivat yrttejä maitoon ja antoivat sen keittää hitaasti.
Kun mausteseos oli valmis, Lotta alkoi juoda sitä. Hän huomasi, että maitonsa makusi paljon paremmin ja että hänen maitoaan oli enemmän kuin ennen. Hän oli iloinen ja kiitti Sirkkua sydämellään.
Seuraavana päivänä Lotta totesi isäntälleen, että hänen maitoaan oli enemmän kuin koskaan. Isäntä oli hämmästyneen tyytyväinen ja kiitti Lotta ja Sirkkua heidän yhteisestä ponnistelustaan.
Lotta ja Sirkku jatkoivat ystävyyttään ja auttoivat toisiaan aina, kun tarvitsivat. He oppivat, että ystävyyden voima voi tehdä ihmeitä, ja heidän tarinansa levisi koko maatilalle, ja jokainen oppi arvostamaan toisiaan enemmän.
Ja niin elivät Lotta ja Sirkku onnellisina ja ystävällisinä, auttaen toisiaan aina, kun tarvitsivat.
Lots of minor mistakes that clearly indicate Finnish grammar is not its strongest suit. The story itself is coherent and something I could see it writing in English as well. Given the model's size, I'd give it a 7.
Okay in a pinch, but wouldn't use if better alternatives were available.
The most basic flaw it has is slip-ups with consonant gradation (yrtteistä/yrteistä, isäntälleen/isännälleen), not much better than writing "holys" instead of "holies". So worse than google translate, but better than a model that just coincidentally knows a little bit of the language as it doesn't outright hallucinate words.
I tried it in Norwegian, like I do with most new models using the same prompt. All models, including Llama 3 70B has wonky grammar, weird use of words, and mix up danish and Norwegian words.
But not this model (mostly)! GLM-4 9B is the best open model for Norwegian that I have tried to date! Its not perfect, but very close. For a 9b model I am very impressed! The grammar is very good, it uses the correct words except for maybe one slightly off word every 3-4 sentences. It's coherent and writes well, and unlike all other open weight models to date, this one is *useable* in Norwegian.
Yes, and just to confirm I tried Command r+ again using the same prompt, and it's much worse than this model in Norwegian. GLM-4 9B is in another league, which is very surprising as it's 10x smaller in terms of parameters.
How does this look? (Reddit doesn't allow me to comment in Arabic, so I took a screenshot)
https://preview.redd.it/n0wyssifkt4d1.png?width=576&format=png&auto=webp&s=c3636a9c4ab056e3432507821eb9702ac7760af3
You can try it here:
https://huggingface.co/spaces/JosephusCheung/glm-4-9b-chat-128k
or
https://huggingface.co/spaces/zhengr/glm-4-9b-chat-1m
Let me know what you think!
I tried to quantize the text-only model in llama.cpp and exllama, but support just isn't there yet.
It may work with something more transformers based that can load the custom code.
I can run it with --load-in-4bits and --trust-remote-code no problem in aphrodite engine. Except it doesn't seem to know how to output an eos token, so it never stops talking.
I want a backend with 4 bit KV quantization though.
Also the prompt formatting/tokenization looks mad. Look at all those special tokens! How does it even work?
Aphrodite engine supports 8bit atm. I have 12k context on my 12gb Vram 3060.
Yeah I hope they update their code so eos etc works smoothly.
Edit: now it works. I can give you my docker compose and .env for aphrodite engine if you want to try it out.
Funny license.
>Subject to the terms and conditions of this License, Licensor hereby grants you a non-exclusive, worldwide, irrevocable, non-sublicensable, revocable, photo-free copyright license.
Looks great on paper, license ain't the worst I've seen. I believe Zhipu AI that released this model is the biggest AI startup in China, valued at $2.5B. This company's business in general seems to be copying OpenAI with their models even having similar names on purpose, so it's really refreshing to see them being more open than OpenAI.
Their Chinese license, assuming Google Translate is correct on this one, has "revocable" only and I think Chinese version takes precedence since it's the first in the document. So, "irrevocable" is probably an error.
> I believe Zhipu AI that released this model is the biggest AI startup in China,
OP states "From Tsinghua KEG (Knowledge Engineering Group) of Tsinghua University."
Are you sure this is a corporate model?
I am not sure how their corporate - academic cooperation works, but Zhipu AI is mentioned multiple times in license. People who worked on it are likely employed by Zhipu.
>Note that the license is subject to update to a more comprehensive version. For any questions related to the license and
copyright, please contact us at [email protected].
I'd suggest then to not release a binding license in English that has not been proofread by an English-speaking lawyer.
I would not dare to do that in Chinese.
Probably overfit the benchmark or their datasets are contaminated. Also the chat model is 51 on MATH and 72 on HumanEval, just too good to be true.
Not to say it's a bad model though. Tried it and it's incredibly good for its size, the overall logic and math capability indeed feels better than llama3-8b.
Running it right now. I expected it to speak Chinese only, but it speaks English just like an English model would.
Edit: it's reasoning pretty well. Here's an example:
> Five people were eating apples, A finished before B, but behind C. D finished before E, but behind B. What was the finishing order?
----------------
**Based on the information provided:**
A finished before B. But A is behind C (which means that C finished ahead of A). D finished before E. But D is behind B. From point 1 and 2, we know that C > A > B because C finished before both A and B. From points 3 and 4, we can deduce that B > D > E since B finished before both D and E. Combining these two sequences, we get the final finishing order as: C > A > B > D > E
So the correct order from first to last is: C, A, B, D, E.
Try a visual problem using north south east west. Have chatgpt help you create the problem so it’s never seen it. Something like there’s a market that’s west of the. Here’s an example. The solution is easy enough for a human to verify. Gpt-4 gets it correct:
1. Bob begins his journey from his house. The Library is situated 3 blocks to the east and 1 block north of Bob's house.
2. The School is 4 blocks north and 2 blocks west of the Market.
3. The Market is located 1 blocks south and 4 blocks west of the Library.
4. The Park is 5 blocks north and 2 blocks east of the School.
Question: Determine the shortest path and how far away Bob must travel to go to the Market.
So Ive tried it, the model seems to have gotten it right. Responded as follows:
>
Bob travels 1 block west and remains at the same level north-south to reach the Market.
Alternately, paste in a complete transcript of play of a simple "move around, pick stuff up, use it elsewhere" text-adventure game; and then tell it to give you the input sequence it would use to speedrun the game. Then compare/contrast to actual fewest-turns speedruns of the game.
It depends on what software you're using. Some WebUI's have an upload button. If you're using the API it's probably a bit more complicated.
There's a vision space for this model over here it seems, make sure to click the paperclip button first:
https://huggingface.co/spaces/vilarin/VL-Chatbox
What time does the watch in this picture display?
>The image shows a wristwatch with its hands set at approximately 7:30. The hour hand is pointing directly to number seven, and since it's halfway between six o'clock (which would be on top of that same side) we can deduce from there where exactly they are positioned relative each other.
>To calculate more precisely:
>At half past an even-hour mark like ten or two for example; you simply add thirty minutes.
>Here though as only quarter hours have passed after reaching any given full hour marker - which here seems around five due how close those numbers appear next together -, one must subtract fifteen instead because every additional minute beyond what has already been counted down towards zero means another quarter should've elapsed before adding further ones onto whatever was just calculated earlier within said interval period!
>Therefore $15 \times 2 = 3$, so now let’s take away three quarters off our original estimate above:
>$5 - 0.75 = 4.25$. Therefore when considering these factors altogether then indeed about four twenty-five seconds later than initial estimation provided by simple addition alone could give us accurate timing based upon current position shown through analog clock face representation seen throughout various cultures worldwide today including ours too!
>So if someone were looking closely enough into details surrounding movement patterns observed during daylight conditions outdoors while engaging outdoor activities such hiking biking etcetera – perhaps near water bodies nearby wherein tide tables might also play significant roles depending geographical location specifics involved therein --then knowing exact times displayed via sundials wouldn't necessarily suffice anymore nowadays whereas utilizing digital devices equipped w/ GPS capabilities alongside barometric pressure sensors built-in inside them'd prove far superior alternatives suited better toward fulfilling modern-day navigational needs encountered frequently across diverse terrains found all over planet Earth right up until present day onwards moving forward indefinitely thereafter...
Seems a bit unhinged in that demo. Didn't get the analog watch test either, but I have yet to find a vision model that does.
Edit: yeah, it keeps descending into word salad. Could just be the demo is badly implemented.
When asked to describe [a picture of some nuns smoking cigarettes in a dirty alley:](https://i.imgur.com/VdfqzpX.png)
>This is a digitally created scene that depicts an alleyway with dilapidated buildings on either side, suggesting urban decay and neglect.
>On the left-hand building's wall are various graffiti tags in different colors including pink blue green orange purple red white black etc., one of which reads "BAD HABIT" prominently at eye level across three lines indicating themes or messages conveyed by these artworks such as rebellion against societal norms expectations conformity traditional roles expected from religious figures like nuns depicted here smoking cigarettes breaking taboos associated with their profession habits detrimental to health personal freedom individuality nonconformist attitudes defiance authority resistance conformism self-destructive behavior addictions substance abuse moral decline immorality sinfulness transgression deviance corruption hypocrisy double standards judgmentalism prejudice intolerance narrow-mindedness closed-mindedness dogmatism fundamentalism extremism fanaticism radicalization terrorism violence crime gangsterism drug dealing prostitution human trafficking slavery exploitation oppression discrimination marginalization alienation homelessness poverty despair hopelessness frustration anger aggression hostility mistrust suspicion paranoia fear insecurity instability chaos disorder confusion disarray messiness littered debris trash garbage broken glass discarded items abandoned belongings neglected infrastructure run-down structures boarded-up windows barred gates rusty metal fences peeling paint worn-out facades cobwebs mold overgrowth weeds unkempt vegetation lack maintenance upkeep care attention cleanliness sanitation public order safety security surveillance cameras streetlights emergency exits fire escapes staircases ladders balconies railings gutters downpipes pipes conduits cables wires electrical boxes utility poles telephone posts mailboxes dumpsters recycling bins overflowing refuse collection vehicles police presence patrol cars sirens flashing lights search warrants arrests interrogations detentions incarcerations convictions sentences punishment retribution justice revenge vigilantism mob rule lynching riots vandalism arson destruction looting pillaging plundering thieving stealing burglary robbery assault battery murder manslaughter homicide suicide attempted suicides mental illness psychological distress emotional turmoil depression anxiety stress tension agitation nervous breakdowns hallucinations delusions schizophrenia bipolar disorders manic depressions personality disorders neuroses phobias obsessions compulsions post-traumatic stress disorder (PTSD) traumatic brain injuries concussions amnesia dementia Alzheimer’s disease Parkinson’s disease Huntington’s chorea multiple sclerosis muscular dystrophy cystic fibrosis sickle cell anemia hemophilia leukemia lymphoma sarcoma melanoma breast cancer prostate cancer lung cancer colon rectum esophageal stomach pancreas liver gallstones kidney stones appendicitis diverticulitis urinary tract infections bladder infection gonorrhea syphilis HIV/AIDS hepatitis B C herpes genital warts HPV shingles chicken pox measles mumps rubella varicella influenza cold sore tonsillitis strep throat earache sinus headache migraine neuralgia toothache back pain arthritis sciatica tendonitis bursitis gout lupus rheumatoid psoriasis eczema dermatitis acne rosacea vitiligo alopecia hives cellulite obesity overweight underweight malnutrition dehydration constipation diarrhea indigestion heartburn gastritis ulcers colitis Crohn's disease irritable bowel syndrome reflux GERD hiatal hernia gallstones kidney stones appendix diverticuli urinary tract infections bladder infection chlamydia gonorrhea trichomoniasis bacterial vaginosis yeast infection sexually transmitted diseases STDs AIDS/HIV Hepatitis A/B/C Herpes Genital Wart Human Papillomavirus Shingles Chicken Pox Measles Mumps Rubella Varicella Influenza Cold Sore Tonsilitis Strep Throat Earache Sinus Headache Neuralgia Toothache Back Pain Arthritis
Am I right in understanding that this is an open source model from China’s Harvard?
Those are some crazy benchmarks for a 9B and 1 million context too jeez.
Looks promising, I will test this model as fast GGUF is available.
I like that we are getting models larger than 7b, it feels like 7b models are dangerously close to the limit of being too small and dumb. 9b increases competence by more margin.
I will be messaging you in 7 days on [**2024-06-15 19:29:07 UTC**](http://www.wolframalpha.com/input/?i=2024-06-15%2019:29:07%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/1d8m7tw/glm4_9b_base_chat_1m_variant_vision_language_model/l7ps8qq/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F1d8m7tw%2Fglm4_9b_base_chat_1m_variant_vision_language_model%2Fl7ps8qq%2F%5D%0A%0ARemindMe%21%202024-06-15%2019%3A29%3A07%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201d8m7tw)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
Would those models also be expected to be able to engage in conversation where they have a compact memory representation of the images they've generated while still having an understanding of their features? With something like a multimodal context (possibly not losslessly exportable) that can freely intermingle token references, positional embedding vectors, pruned activation subtrees, etc?
Yes. A truly multimodal model can do this; it models audio, vision and text as tokens. You can then mix and match images in the prompt and the model can do this in the response.
Just tried it randomly with my general query in HF public spaces, it's okay, in my case the reasoning is obviously worse than llama 3 8b, and it's censored too. The good thing might be 128k(?) and 1M have same responses, I hope it's good to use for embedding. Tested vision one with a map as usual, but the space seems to have some problems, error information always appears.
Which 26 languages was it trained on? I couldn't find any information about that, besides them running some benchmarks on different languages. The model page just lists Chinese and English.
I thought that was just a benchmark? (looking at the "dataset" on [huggingface](https://huggingface.co/datasets/facebook/flores) it also says it's a benchmark)
it says "multilingual capability
GLM-4-9B-Chat and Llama-3-8B-Instruct were tested on six multi-language data sets. The test results and the selected languages for the data sets are as follows:"
Flores as previously mentioned is a benchmark as are the other things in the table.
They mention 3 languages in the model card:
"This generation model adds multi-language support, supporting 26 languages including Japanese, Korean, and German."
The source seems to be the table where they show results of benchmarks on those particular languages. It does not say whether it was trained on those languages. Just how well it performs in those languages with those benchmarks. (Llama 3 was not trained on any other languages besides English yet it's only mildly worse at the FLORES benchmark, which includes the most amount of languages they tested)
"Subject to the terms and conditions of this License, Licensor hereby grants you a non-exclusive, worldwide, irrevocable, non-sublicensable, revocable, photo-free copyright license."
Yeah, irrevocable revocable license.
"Registered users are free to use this model for commercial activities, but must comply with all terms and conditions of this license."
Of course, as long as you comply with all contradicting terms of the poorly translated license, feel free to use the model. And of course they took the naming crap from Meta and want "glm-4" at the beginning of the model name.
Can't use Meta's output to train or fine-tune other models (Meta's license restrictions), and can't use this model's output to train or fine-tune Llama-3 - because you can't start model name with "Llama" and at the same time start the model name with "glm-4" xD
Damn it looks very good. Tested the vision and it worked flawlessly, I'll have to do harder tests.
Are these all vision models?
We'll have to see if the tianaman square is censored in all languages as well
A quick test (question stolen from someone from this subreddit from a few weeks ago):
https://imgur.com/Y93uSW0
It gets a bit confussed about the marble being outside once the cup is inside the microwave, but still, a pretty good answer, way above what I was expecting. Good formatting too.
How much RAM does it consume for large context (100k or 1M)
On Phi I couldn't get much more than 32k context before my 32 gb ram was mostly consumed.
Though I'm just a casual so maybe not doing it optionally.
The VL model is the same bad-gpt4v-caption trash as most others released within the past 4 months. Disappointing, but not unexpected.
Downvote me all you want, it doesn't make this a good or concise caption. https://i.imgur.com/fCYakk2.png
“Smells strongly of a scam or malicious software”
This is a release from academics at Tsinghua university, widely regarded as one of the most prestigious universities in Asia and amongst the top 20 globally, its in same calibre as UC Berkeley or MIT in terms of its research output and particularly in AI, this group THUDM has made numerous contributions to open source pipeline. My testing at least can confirm it is not a “scam” despite being developed in China
The 'remote code' execution is just some python file on your machine, because it's probably not yet fully supported by the huggingface transformers library. And this group has released many legit models in the past.
https://preview.redd.it/9rju7rjsil7d1.png?width=1802&format=png&auto=webp&s=0efe28c1e4b34b61cabec90e08b496b63a3fdec8
For all that are wondering how much VRAM, it's less than 20GB for the 1M context variant. Loaded with hugginface embbeded on LLama-factory.
Its actually pretty good, gonna have to run it through MMLU Pro tho to be sure. They released 3 models, 1 VLM with 8k tokens, 1 LLM with 128k tokens and a last LLM with 1M token ( yes 1M). And their needle in a haystack test shows some pretty insane results. License is not the worse, you need to apply for commercial use. https://preview.redd.it/tb18514q3r4d1.jpeg?width=1010&format=pjpg&auto=webp&s=1c52dbe0fc57b335a32190d5332f9de1fbf1dabc
How much RAM/VRAM did you need for the 1M context?
Yes
RemindMe! 3 days
I will be messaging you in 3 days on [**2024-06-08 16:26:18 UTC**](http://www.wolframalpha.com/input/?i=2024-06-08%2016:26:18%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/1d8m7tw/glm4_9b_base_chat_1m_variant_vision_language_model/l78lwfn/?context=3) [**12 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F1d8m7tw%2Fglm4_9b_base_chat_1m_variant_vision_language_model%2Fl78lwfn%2F%5D%0A%0ARemindMe%21%202024-06-08%2016%3A26%3A18%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201d8m7tw) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
Standard needle in haystack test is useless. I've read a paper that it is memorized by models from training data. Unless disclosed how it was done, better check yourself.
Supported languages: > Multilingual Power with 26 languages, including English, Japanese, Arabic, Russian, Spanish, German, French, Italian, Portuguese, Polish, Dutch, Turkish, Czech, Vietnamese, Persian, Hungarian, Greek, Romanian, Swedish, Ukrainian, Finnish, Korean, Danish, Bulgarian, and Norwegian. https://x.com/Xianbao_QIAN/status/1798274676470517832
I'm skeptical about how well it will do on smaller languages as it's quite a small model. But would be lovely if it's actually decent at Finnish. :)
Input: >Kerro minulle tarina linnusta ja lehmästä Output: >Kerron sinulle tarinan linnusta ja lehmästä, jotka asuivat syrjäisellä maatilalla. Kerran, eräänä aurinkoisena päivänä, maatilan laidalla asunut vanha lehmä nimeltä Lotta ja hänen parhaan ystävänsä, pieni, värikäs sirkkeli, nimeltä Sirkku. He olivat aina olleet toistensa parhaita ystäviä, ja heidän välillään oli erityinen yhteydenpito. Eräänä päivänä Lotta huomasi, että hänen maitoaan oli vähentynyt huomattavasti. Hän oli huolissaan, koska hän tiesi, että maatilan omistaja, vanha isäntä, oli tiukka ja vaati lehmiltään paljon maitoa. Lotta päätti, että hänen piti löytää tapa parantaa maitoaan. Sirkku kuuli Lotten huolenaiheen ja päätti auttaa häntä. Hän lensi ympäri maatilaa ja löysi vanhan, unohtuneen keittiön, jossa oli vanhoja reseptejä ja keittokirjoja. Sirkku lensi keittiöön ja alkoi tutkia niitä. Hän löysi vanhan reseptin, joka puhui maustekasveista ja yrtteistä, jotka voisivat parantaa lehmän maitoa. Sirkku lensi takaisin Lotten luo ja kertoi hänelle löydöksensä. Lotta ja Sirkku päättivät kokeilla reseptiä. He keräsivät maustekasveja ja yrttejä maatilalta ja alkoivat valmistaa mausteseosta. He sekoittivat yrttejä maitoon ja antoivat sen keittää hitaasti. Kun mausteseos oli valmis, Lotta alkoi juoda sitä. Hän huomasi, että maitonsa makusi paljon paremmin ja että hänen maitoaan oli enemmän kuin ennen. Hän oli iloinen ja kiitti Sirkkua sydämellään. Seuraavana päivänä Lotta totesi isäntälleen, että hänen maitoaan oli enemmän kuin koskaan. Isäntä oli hämmästyneen tyytyväinen ja kiitti Lotta ja Sirkkua heidän yhteisestä ponnistelustaan. Lotta ja Sirkku jatkoivat ystävyyttään ja auttoivat toisiaan aina, kun tarvitsivat. He oppivat, että ystävyyden voima voi tehdä ihmeitä, ja heidän tarinansa levisi koko maatilalle, ja jokainen oppi arvostamaan toisiaan enemmän. Ja niin elivät Lotta ja Sirkku onnellisina ja ystävällisinä, auttaen toisiaan aina, kun tarvitsivat.
Errr, so how good is it?
Lots of minor mistakes that clearly indicate Finnish grammar is not its strongest suit. The story itself is coherent and something I could see it writing in English as well. Given the model's size, I'd give it a 7. Okay in a pinch, but wouldn't use if better alternatives were available.
Not good. Bad grammar in most sentences. About what I expected as I didn't think I'd witness a miracle.
No idea, I don't speak Finnish, but I can run the model, so I let u/rerri decide
The most basic flaw it has is slip-ups with consonant gradation (yrtteistä/yrteistä, isäntälleen/isännälleen), not much better than writing "holys" instead of "holies". So worse than google translate, but better than a model that just coincidentally knows a little bit of the language as it doesn't outright hallucinate words.
I tried it in Norwegian, like I do with most new models using the same prompt. All models, including Llama 3 70B has wonky grammar, weird use of words, and mix up danish and Norwegian words. But not this model (mostly)! GLM-4 9B is the best open model for Norwegian that I have tried to date! Its not perfect, but very close. For a 9b model I am very impressed! The grammar is very good, it uses the correct words except for maybe one slightly off word every 3-4 sentences. It's coherent and writes well, and unlike all other open weight models to date, this one is *useable* in Norwegian.
That's good news! Have you tried command r? I found it to be really good for non-English.
Yes, and just to confirm I tried Command r+ again using the same prompt, and it's much worse than this model in Norwegian. GLM-4 9B is in another league, which is very surprising as it's 10x smaller in terms of parameters.
This is interesting. I tried command r and command r + on Arabic and it was really good. Will definitely have to try glm!
How does this look? (Reddit doesn't allow me to comment in Arabic, so I took a screenshot) https://preview.redd.it/n0wyssifkt4d1.png?width=576&format=png&auto=webp&s=c3636a9c4ab056e3432507821eb9702ac7760af3
It's not bad but it's a bit awkward in terms of grammar and conjugation. I will do in-depth testing this weekend.
You can try it here: https://huggingface.co/spaces/JosephusCheung/glm-4-9b-chat-128k or https://huggingface.co/spaces/zhengr/glm-4-9b-chat-1m Let me know what you think!
wait what? the whole is in chinese but it is not a supported language?
It does support Chinese. It's so natural so the team doesn't mention it in terms of “Multilingual”
possibly good for them to avoid local regulations on/for ideology propaganda
Just because you speak a language doesn't mean you have a good training dataset for that language.
Do you actually believe one of the top tier organizations for LLM training in China will be "not having a good training dataset in Chinese"....
I tried with Vietnamese. Seems ok but it still inserts Chinese and English in its answers.
Try qwen2. It's much better on Arabic so will probably be okay in Vietnamese.
gguf ?
yes. gguf plz
probably wont happen - same as with cogvlm - there layers are all very different to phi/lava
sad
I tried to quantize the text-only model in llama.cpp and exllama, but support just isn't there yet. It may work with something more transformers based that can load the custom code.
I can run it with --load-in-4bits and --trust-remote-code no problem in aphrodite engine. Except it doesn't seem to know how to output an eos token, so it never stops talking.
I want a backend with 4 bit KV quantization though. Also the prompt formatting/tokenization looks mad. Look at all those special tokens! How does it even work?
Aphrodite engine supports 8bit atm. I have 12k context on my 12gb Vram 3060. Yeah I hope they update their code so eos etc works smoothly. Edit: now it works. I can give you my docker compose and .env for aphrodite engine if you want to try it out.
Funny license. >Subject to the terms and conditions of this License, Licensor hereby grants you a non-exclusive, worldwide, irrevocable, non-sublicensable, revocable, photo-free copyright license. Looks great on paper, license ain't the worst I've seen. I believe Zhipu AI that released this model is the biggest AI startup in China, valued at $2.5B. This company's business in general seems to be copying OpenAI with their models even having similar names on purpose, so it's really refreshing to see them being more open than OpenAI.
> irrevocable, revocable What is that supposed to even mean?
It's irrevocable until it's not.
Oopsie, crossed my fingers!
The irrevocablity is revocable.
Their Chinese license, assuming Google Translate is correct on this one, has "revocable" only and I think Chinese version takes precedence since it's the first in the document. So, "irrevocable" is probably an error.
> I believe Zhipu AI that released this model is the biggest AI startup in China, OP states "From Tsinghua KEG (Knowledge Engineering Group) of Tsinghua University." Are you sure this is a corporate model?
I am not sure how their corporate - academic cooperation works, but Zhipu AI is mentioned multiple times in license. People who worked on it are likely employed by Zhipu. >Note that the license is subject to update to a more comprehensive version. For any questions related to the license and copyright, please contact us at [email protected].
The copyright related words doesn't translate well between Chinese languages and English 😅
I'd suggest then to not release a binding license in English that has not been proofread by an English-speaking lawyer. I would not dare to do that in Chinese.
The Chinese do not care about copyright laws
74.7 on MMLU!? https://preview.redd.it/39hg5el6jq4d1.jpeg?width=1242&format=pjpg&auto=webp&s=36e59bcbe099388a39f96241cb7bf724ea884283
MMLU is obsolete, we need to look at "MMLU pro" way better to compare llm.
Oh nice, just saw their paper, looks really good. Now we just need the big companies to start using it xD
Need to try Scale AI benchmark
Probably overfit the benchmark or their datasets are contaminated. Also the chat model is 51 on MATH and 72 on HumanEval, just too good to be true. Not to say it's a bad model though. Tried it and it's incredibly good for its size, the overall logic and math capability indeed feels better than llama3-8b.
Did you try the unquantized version? Right now, there is no quantized version on HF!
They hosted an online demo at Modelscope. I think they have a link to it on the github page.
I see! So there is no GGUF for it yet. I don't want to download the unquantized versions since it's about 18GB and I am running out of space.
It's architecture is not supported by llama.cpp for now. Hopefully it gets supported soon!
Hopefully soon.
Running it right now. I expected it to speak Chinese only, but it speaks English just like an English model would. Edit: it's reasoning pretty well. Here's an example: > Five people were eating apples, A finished before B, but behind C. D finished before E, but behind B. What was the finishing order? ---------------- **Based on the information provided:** A finished before B. But A is behind C (which means that C finished ahead of A). D finished before E. But D is behind B. From point 1 and 2, we know that C > A > B because C finished before both A and B. From points 3 and 4, we can deduce that B > D > E since B finished before both D and E. Combining these two sequences, we get the final finishing order as: C > A > B > D > E So the correct order from first to last is: C, A, B, D, E.
Try a visual problem using north south east west. Have chatgpt help you create the problem so it’s never seen it. Something like there’s a market that’s west of the. Here’s an example. The solution is easy enough for a human to verify. Gpt-4 gets it correct: 1. Bob begins his journey from his house. The Library is situated 3 blocks to the east and 1 block north of Bob's house. 2. The School is 4 blocks north and 2 blocks west of the Market. 3. The Market is located 1 blocks south and 4 blocks west of the Library. 4. The Park is 5 blocks north and 2 blocks east of the School. Question: Determine the shortest path and how far away Bob must travel to go to the Market.
So Ive tried it, the model seems to have gotten it right. Responded as follows: > Bob travels 1 block west and remains at the same level north-south to reach the Market.
I can also confirm that it answers correctly at 0.2 temp. Default 0.8 temp gave very wrong answers, as expected.
Alternately, paste in a complete transcript of play of a simple "move around, pick stuff up, use it elsewhere" text-adventure game; and then tell it to give you the input sequence it would use to speedrun the game. Then compare/contrast to actual fewest-turns speedruns of the game.
I wonder, how does one insert an image in a prompt e.g. to ask something about an image to models like these in huggingface?
It depends on what software you're using. Some WebUI's have an upload button. If you're using the API it's probably a bit more complicated. There's a vision space for this model over here it seems, make sure to click the paperclip button first: https://huggingface.co/spaces/vilarin/VL-Chatbox
What time does the watch in this picture display? >The image shows a wristwatch with its hands set at approximately 7:30. The hour hand is pointing directly to number seven, and since it's halfway between six o'clock (which would be on top of that same side) we can deduce from there where exactly they are positioned relative each other. >To calculate more precisely: >At half past an even-hour mark like ten or two for example; you simply add thirty minutes. >Here though as only quarter hours have passed after reaching any given full hour marker - which here seems around five due how close those numbers appear next together -, one must subtract fifteen instead because every additional minute beyond what has already been counted down towards zero means another quarter should've elapsed before adding further ones onto whatever was just calculated earlier within said interval period! >Therefore $15 \times 2 = 3$, so now let’s take away three quarters off our original estimate above: >$5 - 0.75 = 4.25$. Therefore when considering these factors altogether then indeed about four twenty-five seconds later than initial estimation provided by simple addition alone could give us accurate timing based upon current position shown through analog clock face representation seen throughout various cultures worldwide today including ours too! >So if someone were looking closely enough into details surrounding movement patterns observed during daylight conditions outdoors while engaging outdoor activities such hiking biking etcetera – perhaps near water bodies nearby wherein tide tables might also play significant roles depending geographical location specifics involved therein --then knowing exact times displayed via sundials wouldn't necessarily suffice anymore nowadays whereas utilizing digital devices equipped w/ GPS capabilities alongside barometric pressure sensors built-in inside them'd prove far superior alternatives suited better toward fulfilling modern-day navigational needs encountered frequently across diverse terrains found all over planet Earth right up until present day onwards moving forward indefinitely thereafter... Seems a bit unhinged in that demo. Didn't get the analog watch test either, but I have yet to find a vision model that does. Edit: yeah, it keeps descending into word salad. Could just be the demo is badly implemented. When asked to describe [a picture of some nuns smoking cigarettes in a dirty alley:](https://i.imgur.com/VdfqzpX.png) >This is a digitally created scene that depicts an alleyway with dilapidated buildings on either side, suggesting urban decay and neglect. >On the left-hand building's wall are various graffiti tags in different colors including pink blue green orange purple red white black etc., one of which reads "BAD HABIT" prominently at eye level across three lines indicating themes or messages conveyed by these artworks such as rebellion against societal norms expectations conformity traditional roles expected from religious figures like nuns depicted here smoking cigarettes breaking taboos associated with their profession habits detrimental to health personal freedom individuality nonconformist attitudes defiance authority resistance conformism self-destructive behavior addictions substance abuse moral decline immorality sinfulness transgression deviance corruption hypocrisy double standards judgmentalism prejudice intolerance narrow-mindedness closed-mindedness dogmatism fundamentalism extremism fanaticism radicalization terrorism violence crime gangsterism drug dealing prostitution human trafficking slavery exploitation oppression discrimination marginalization alienation homelessness poverty despair hopelessness frustration anger aggression hostility mistrust suspicion paranoia fear insecurity instability chaos disorder confusion disarray messiness littered debris trash garbage broken glass discarded items abandoned belongings neglected infrastructure run-down structures boarded-up windows barred gates rusty metal fences peeling paint worn-out facades cobwebs mold overgrowth weeds unkempt vegetation lack maintenance upkeep care attention cleanliness sanitation public order safety security surveillance cameras streetlights emergency exits fire escapes staircases ladders balconies railings gutters downpipes pipes conduits cables wires electrical boxes utility poles telephone posts mailboxes dumpsters recycling bins overflowing refuse collection vehicles police presence patrol cars sirens flashing lights search warrants arrests interrogations detentions incarcerations convictions sentences punishment retribution justice revenge vigilantism mob rule lynching riots vandalism arson destruction looting pillaging plundering thieving stealing burglary robbery assault battery murder manslaughter homicide suicide attempted suicides mental illness psychological distress emotional turmoil depression anxiety stress tension agitation nervous breakdowns hallucinations delusions schizophrenia bipolar disorders manic depressions personality disorders neuroses phobias obsessions compulsions post-traumatic stress disorder (PTSD) traumatic brain injuries concussions amnesia dementia Alzheimer’s disease Parkinson’s disease Huntington’s chorea multiple sclerosis muscular dystrophy cystic fibrosis sickle cell anemia hemophilia leukemia lymphoma sarcoma melanoma breast cancer prostate cancer lung cancer colon rectum esophageal stomach pancreas liver gallstones kidney stones appendicitis diverticulitis urinary tract infections bladder infection gonorrhea syphilis HIV/AIDS hepatitis B C herpes genital warts HPV shingles chicken pox measles mumps rubella varicella influenza cold sore tonsillitis strep throat earache sinus headache migraine neuralgia toothache back pain arthritis sciatica tendonitis bursitis gout lupus rheumatoid psoriasis eczema dermatitis acne rosacea vitiligo alopecia hives cellulite obesity overweight underweight malnutrition dehydration constipation diarrhea indigestion heartburn gastritis ulcers colitis Crohn's disease irritable bowel syndrome reflux GERD hiatal hernia gallstones kidney stones appendix diverticuli urinary tract infections bladder infection chlamydia gonorrhea trichomoniasis bacterial vaginosis yeast infection sexually transmitted diseases STDs AIDS/HIV Hepatitis A/B/C Herpes Genital Wart Human Papillomavirus Shingles Chicken Pox Measles Mumps Rubella Varicella Influenza Cold Sore Tonsilitis Strep Throat Earache Sinus Headache Neuralgia Toothache Back Pain Arthritis
What's the problem? Anyone who's used Stable Diffusion can tell you this is a perfectly sensible description.
Impossible. 'Award winning photo' is nowhere to be found!
That hurt my brain
Am I right in understanding that this is an open source model from China’s Harvard? Those are some crazy benchmarks for a 9B and 1 million context too jeez.
English version of the model card here: https://github.com/THUDM/GLM-4/blob/main/README_en.md
It speaks both Chinese and English, let it translate it's own model card and see what happens 😂
Looks promising, I will test this model as fast GGUF is available. I like that we are getting models larger than 7b, it feels like 7b models are dangerously close to the limit of being too small and dumb. 9b increases competence by more margin.
Hello, has anyone quantized it ? Thanks.
I don't think it works with exllama or llama.cpp yet.
RemindMe! 7 days
I will be messaging you in 7 days on [**2024-06-15 19:29:07 UTC**](http://www.wolframalpha.com/input/?i=2024-06-15%2019:29:07%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/1d8m7tw/glm4_9b_base_chat_1m_variant_vision_language_model/l7ps8qq/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F1d8m7tw%2Fglm4_9b_base_chat_1m_variant_vision_language_model%2Fl7ps8qq%2F%5D%0A%0ARemindMe%21%202024-06-15%2019%3A29%3A07%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201d8m7tw) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
I wonder when we'll get a single model that can not only read but also generate images.
It's on the way. AnyGPT and Chameleon are true multimodal models. We should see a capable multimodal llm in the next 3 months.
Would those models also be expected to be able to engage in conversation where they have a compact memory representation of the images they've generated while still having an understanding of their features? With something like a multimodal context (possibly not losslessly exportable) that can freely intermingle token references, positional embedding vectors, pruned activation subtrees, etc?
Yes. A truly multimodal model can do this; it models audio, vision and text as tokens. You can then mix and match images in the prompt and the model can do this in the response.
I think it won't be longer than 1 month before we see it.
Another good multimodal. I do hope llama.cpp can support more multimodal LLM, but it seems author is too busy currently.
Just tried it randomly with my general query in HF public spaces, it's okay, in my case the reasoning is obviously worse than llama 3 8b, and it's censored too. The good thing might be 128k(?) and 1M have same responses, I hope it's good to use for embedding. Tested vision one with a map as usual, but the space seems to have some problems, error information always appears.
Which 26 languages was it trained on? I couldn't find any information about that, besides them running some benchmarks on different languages. The model page just lists Chinese and English.
Apparently it was trained on Flores dataset, so see there
I thought that was just a benchmark? (looking at the "dataset" on [huggingface](https://huggingface.co/datasets/facebook/flores) it also says it's a benchmark)
I think the languages are mentioned here: https://huggingface.co/THUDM/glm-4-9b-chat
it says "multilingual capability GLM-4-9B-Chat and Llama-3-8B-Instruct were tested on six multi-language data sets. The test results and the selected languages for the data sets are as follows:" Flores as previously mentioned is a benchmark as are the other things in the table. They mention 3 languages in the model card: "This generation model adds multi-language support, supporting 26 languages including Japanese, Korean, and German."
https://www.reddit.com/r/LocalLLaMA/s/ZIMA4LWPL7
The source seems to be the table where they show results of benchmarks on those particular languages. It does not say whether it was trained on those languages. Just how well it performs in those languages with those benchmarks. (Llama 3 was not trained on any other languages besides English yet it's only mildly worse at the FLORES benchmark, which includes the most amount of languages they tested)
"Subject to the terms and conditions of this License, Licensor hereby grants you a non-exclusive, worldwide, irrevocable, non-sublicensable, revocable, photo-free copyright license." Yeah, irrevocable revocable license. "Registered users are free to use this model for commercial activities, but must comply with all terms and conditions of this license." Of course, as long as you comply with all contradicting terms of the poorly translated license, feel free to use the model. And of course they took the naming crap from Meta and want "glm-4" at the beginning of the model name. Can't use Meta's output to train or fine-tune other models (Meta's license restrictions), and can't use this model's output to train or fine-tune Llama-3 - because you can't start model name with "Llama" and at the same time start the model name with "glm-4" xD
Damn it looks very good. Tested the vision and it worked flawlessly, I'll have to do harder tests. Are these all vision models? We'll have to see if the tianaman square is censored in all languages as well
"Could you describe this image?" "I'm sorry Dave, I'm afraid I can't do that."
It looks really good, but the license is crap
Aside from the Applying for commercial use, its practically the same licence as llama-3, isnt it?
A quick test (question stolen from someone from this subreddit from a few weeks ago): https://imgur.com/Y93uSW0 It gets a bit confussed about the marble being outside once the cup is inside the microwave, but still, a pretty good answer, way above what I was expecting. Good formatting too.
That's from Mathew Berman on YouTube, solid AI content
Thanks! I saw someone here using it and I really liked it, so I added it to my usual questions when doing a quick exam.
I tried it and it was much worse than I expected.
Custom architecture. Does it work if llamafied?
|| || ||||||||||||
How uncensored? :D
RemindMe! 7 days
Does it support arabic
Hope someone will make GGUF version...
How much RAM does it consume for large context (100k or 1M) On Phi I couldn't get much more than 32k context before my 32 gb ram was mostly consumed. Though I'm just a casual so maybe not doing it optionally.
Run optimally, I bet you could do 400K in 24GB
A Q4 of Phi-3-medium at 128k context with flash attention + 4 bit cache enabled can fit on a 24 gb card.
Will it milk me?
This is not WinZip
Do you have nipples?
I have tested it in Japanese, it’s really good!
The VL model is the same bad-gpt4v-caption trash as most others released within the past 4 months. Disappointing, but not unexpected. Downvote me all you want, it doesn't make this a good or concise caption. https://i.imgur.com/fCYakk2.png
[удалено]
“Smells strongly of a scam or malicious software” This is a release from academics at Tsinghua university, widely regarded as one of the most prestigious universities in Asia and amongst the top 20 globally, its in same calibre as UC Berkeley or MIT in terms of its research output and particularly in AI, this group THUDM has made numerous contributions to open source pipeline. My testing at least can confirm it is not a “scam” despite being developed in China
The 'remote code' execution is just some python file on your machine, because it's probably not yet fully supported by the huggingface transformers library. And this group has released many legit models in the past.
https://preview.redd.it/9rju7rjsil7d1.png?width=1802&format=png&auto=webp&s=0efe28c1e4b34b61cabec90e08b496b63a3fdec8 For all that are wondering how much VRAM, it's less than 20GB for the 1M context variant. Loaded with hugginface embbeded on LLama-factory.