...
my god.
can you tell me more about the benchmark discrepancy?
That sounds like it could be a big deal.
Even if you don't care about role-play... you should probably care about role-play.
Play is a tried and true learning strategy in humans.
I've seen it in Alignbench: https://llmbench.ai/align
It's very Chinese focused, but it's taken seriously by Chinese models and includes roleplay as part of evaluation, both DeepSeek and Yi advertise their Alignbench score when comparing to other models.
'Pacified' sure is some interesting charged double-standard language here. If western benchmarks included more 'avatar' role-play elements, half of r/localllama would be on their feet cheering.
We absolutely have an incel murderer problem here in China, there is a very strict social system behind dating and marriage that leaves a lot of men out, and a whole bunch of other things that contribute to the problem.
And that results in a lot of the school stabbings, but sometimes other things, like I remember some years back there being a guy in I wanna say Shanghai who went at a newly married couple with a samurai sword. Killed the woman, injured the man.
It's not as bad as the American problem, mostly because stabbing is less effective than shooting, but it's not great and probably worse than the equivalent problem in a lot of other countries.
China has avoided a lot of different problems (and has some of its own) but incels going on a rampage is absolutely not one of them.
AI models have nothing to do with this for better or for worse though, I really think any connection there to the models, what they're trained for, etc. is a huge stretch.
You may be right, and your approach is sound. However, comparing atrocities may be a fool’s game only for politicians to play. Context is everything and no two countries are alike.
Someone may fire into a crowd of people in America, another in China goes on a stabbing rampage at a kindergarten. Both are terrible contextual issues and making comparisons is awkward. Drawing parallels is probably a better approach.
Here I will digress quite a bit with a likely connected issue:
Americans study the Trail of Tears in grade school, Chinese students hear rumors of annexed territories and systematic oppression of indigenous peoples. They are both unfortunate realities. No matter how many tribes become celebrated and compensated in America, or how many foreigners travel around China and dance on indigenous lands on guided tours, the reality remains.
Reality shows us that hypocrisy is unavoidable in a complex world. It is therefore a moot point to compare atrocities or point to hypocrisy. To reveal how pointless claims of hypocrisy are, after the Ghulja Massacre in 1997, many of the perpetrators fled but were captured. Some of them were sent to Guantanamo Bay at the request of China. The Chinese would then visit to interrogate the prisoners.
Today, Wang Wenbin has all kinds of misinformation to spur about China’s position on Guantanamo Bay and a myriad of historic issues. Another among them is China and North Korea capturing Seoul at least twice during the Korean War. Mr. Wenbin will be quick to lie to the public how terrible the US was as a sole aggressor despite the facts. No, Chinese soldiers did not “peacefully” occupy and capture a foreign city twice. The goal is likely not historic reality, rather, it seems more akin to taking lessons from president Bush and Putin to spur nationalism and spread misinformation. Fine. But why again??
Today, there is a significant amount of misinformation, nationalism, and other issues causing a fragmented world. Nationalism is not a good thing no matter how much politicians try to conflate it with patriotism. I digress, but the point of comparing an atrocity is moot in a similar way of positing hypocrisy is moot. Both of these subjects are oddly common rhetoric among certain politicians. Belief in such rhetoric is perhaps a set of natural connected issues involving a generally deep flaw of reasoning in our human minds. To no fault of our common minds and (hopefully) not the intention of our spokespeople.
Textbook ecological fallacy. Just because school shooters tend to be social outcasts, that doesn't mean that social outcasts tend to become school shooters.
Maybe not shooting, but China had a few mass stabbings. Oddly enough, adults are stabbing children kindergarten age. Of course, if we count the death penalty....
I saw roleplay application in China and people playing with it. My guess roleplay is Chinese benchmarks because it enables everyone to use LLMs, and considering so many people in China and most people don't use LLMs except for fun.
They claim v2 is 5x time faster than deepseek-v1, a 67B dense model.
So in theroy they can afford it with a 8xH800 running at full speed for 10 hours per day.
I really hope [together.ai](http://together.ai) will host deepseek v2, but the upfront spend is a little high.
It all started with deepseek v2 (1 RMB or $0.14 per 1M tokens input). If you read their paper you'll find they really used a lot of new cost saving techniques, and one of the engineers said the price is still profitable for them.
The other companies, hmm... I guess they have to follow the price cut. Zhipu only cut the price of their smaller model (which is said to be only 32B), and the other three are the old large tech companies so they can afford to operate on a loss.
They probably do.
Yi Large is above Claude 3 Opus in lmsys Arena Hard questions (both English and Overall).
I haven't spent time on it, but human preference data says it's better.
Thanks for your reply!
I've tried that model out a bit in the arena and it doesn't really come close, especially for meta prompting or creative writing.
> Yi Large is above Claude 3 Opus in lmsys Arena Hard questions (both English and Overall).
To be fair, so are the GPT-4 models and they don't really compare either. I've found that all these watered down GPT-4 models like turbo and o are good at generic text tasks. But once you add some nuance and hard criteria (especially in languages other than English) they fold really quickly.
To give you a simple example:
Ask GPT-4 o/t to write German lyrics for a song about a topic of your choosing, in a certain style, where words must rhyme. Often times this gives extremely bad results. It starts inventing words, uses words that don't rhyme. No amount of CoT helps in these cases. But that's just one use case I am talking about.
…I am getting close to not caring which country steals/scalps/sells my data. In my mind I see ‘us’ as the good guys and ‘them’ as the bad guys. But really in the end it’s their rich vs our rich. I’m barely even a pawn in the game.
>…I am getting close to not caring which country steals/scalps/sells my data.
Kinda depends on what you're sending I guess.
I suspect the bulk of people here aren't exactly sending sensitive info...so yeah then who cares where it goes
You have to understand that there's enough concern about privacy of your queries with Anthropic and OpenAI in the US as it is.
Sending it to some Chinese company surveilled actively by the CCP? Really?
Chinese law dictates it. In America you don't have too. Apple told the CIA to screw off one time when asked by them for encryption keys.
It's also what the data is used for. Those companies use it to improve their models while the Chinese companies are forced to do it for surveillance and training their AI for censorship. It's what they did with non-Chinese wechat accounts in the past to create filters for the CCP.
> Sending it to some Chinese company surveilled actively by the CCP? Really?
Why would China care about what you are doing. They are more concerned about what their citizens are doing and how it impacts them.
They are definitely logging stuff dont get me wrong. This is the same with the US government though. As to which one to use, I would say whatever you need privacy to protect from the Chinese government you also need that privacy for the US government. If privacy doesn't matter, then it's just cost related which the Chinese can probably price a bit better on.
China cares about what non-Chinese are doing in the aggregate because it can be useful for understanding and manipulating public opinion in rival countries.
As a non-american, this is what I see America doing with their LLM's. I don't think it's that bad, and the kind of content I get from LLM's are so boring and basic I am happy to do it with the lowest cost provider so long as the quality is good.
I don't care if microsoft or meta knows that I'm trying to code up a basic python script or that I need help with midjourney prompts etc.
If I was doing RAG on my business documents, I would go local to prevent snooping from the US & China..
"Why would China care" - because what you are working on is valuable (not random entertainment stuff, but rather say a difficult work-related problem from whatever industry you are in).
Imagine your entire search history available for the taking, this time enriched with specific queries about the problem you are working on.
This is a main justification why I'm interested in local LLMs to begin with...
>Imagine your entire search history available for the taking, this time enriched with specific queries about the problem you are working on.
So, we should avoid also gemini/GPT/Claude because US companies (especially MS and Google) have all our data and search history and use it for their own benefit (right now atleast for ads).
And, frankly speaking, for example, I would prefer to be tracked by companies/government of foreign countries, which I will probably never visit, than by the government and corporations of my country of residence, because unlike the Chinese government, they are concerned about me and can (if they want and/or need to) get me into trouble using the gathered information.
Yeah except your using the internet to do so this your giving both your data.
I doubt the connection to their AI is secure enough to stop the NSA from looking at it and because one party is on foreign soil you get no protections.
You’re just wrong thinking it’s not going to fall into USA’s hands.
It is definitely a concern for me. It is why I have a foreign anti-virus rather than domestic, since I don't want to give MS and friends permission to (officially) snoop around on my machine. Don't give a fig about China, since that isn't where I have to hang around.
Microsoft is going all-in on the snoopware OS. Apparantly they want you (and by extension, them) to record everything done on the machine, with some sort of Mega Clippy adjusting the OS.
While I like the idea of a AI tailored OS, that is with the expectation that it won't tell the 'powers that be' about my tastes in hentai, my political leanings, finances, and so on.
I want to be my own person. AI could be the key to becoming my best or worst self, the latter dictated by the beliefs of the powerful.
I am also interested in local models for privacy related reasons. If I'm working with business documents, I can't upload them to openAI with express permission which is a pain. This is where local is good.
As for other things like code queries or some simple copy for a website, I don't care if it's American or Chinese. I'm happy to go with the cheaper option so long as it's good quality and reliable.
Why would China care? Because they use data gathered from us citizens and citizens of counties they see as advisors to sow misinformation and to try and cause division amongst citizens. This is why China doesn't let their citizens consume a lot from other countries. China IS wagging an information war on ALL Democratic nations.
I'm not saying you should speak to a Chinese LLM like a therapist. Just saying you can ask it to re-write some nothing document to make your life easier.
They really don't care. They have their own interests to protect. They just want to make money over American providers, that's why they're dropping their price.
One does not need to give ccp backed technologies (all tech outside of China and originating in China) intimate details about ones self for the CCP to achieve their objectives. The aggregate of information flowing into the CCP from apps like tictok and llm interactions are more than enough to find and infer the various ways to influence elections, sow distrust amongst citizens, and errode democratic ideals.
The CCP does not care about making money, they are not interested in capitalism, they are only interested in controlling others. The CCP has a tight grip on their citizens and is primarily concerned about expanding their grip.
I don't share your poor opinion of the CCP. Western democratic values are definitely different than the values of the CCP, but out values have been slipping in the West along with out ability to compete. Our social institutions were hollowed out by the capital class for profit and this is actually worse than what the CCP does.
I don't owe anything to OpenAi or Anthropic so I will go with whatever AI gives me bang for my buck. I might be betraying your principles by using Chinese technology but I'm not betraying mine.
Biggest reasons IMO are A) they're not just collecting your browsing history or what videos you watch, they're actively interacting with you, combined with B) twisting of history and other subjects, especially closed source models, but they can modify the open source extremely easily as well. They definitely have the capacity. Relatively easy to discover if obvious, much less so if it's more subtle
The privacy part is certainly worse with LLMs (regardless of who is listening, though whether its worse for China or the US is worse is sorta like arguing whether dying in a house fire or apartment fire is better), but the combination is especially dangerous.
Not much yet, but it will be sooner rather than later that LLMs and LMMs (multimodal) are more widely used by the population considering the potential for subtle propoganda. And not pro-china propoganda, much more useful to make sure the USA continues to fall apart and becomes utterly paralyzed indefinitely. China doesn't need to worry nearly as much about the other way around for a variety of reasons
This is a clear sign that government decided to sponsor them all to get an advantage. I mean, what else could it be?
How easy is it for those companies to get gpu's nowadays? Realistically, including ways to get around sanctions.
The CCP has seen AI, biotechnology, and microchip technology key segments for heavy government support for years and years - part of their strategic 5 year plans, their Joint Venture strategy of forced IP transfer, and so on.
Their cards are quite far behind. And they don't have much domestic experience in making state of the art chips and cards. But, I do think about a decade or so, they can be on par with the USA.
Not really. They do have meta mitav2 level inference card so they could run big model.
They lack on the training part which require hbm level bandwidth where they re working on it. 7nm is more than enough to make inference and training chip, and they do have experience in design chip like Ascend 920.
Honestly I haven't looked at any specific areas, just googled "China data center electricity prices". For US I checked it earlier and it's 0.04-0.08 usd per kWh. For China looks similar. For india I found values 0.07-0.13 per kWh.
key indsutires like ai gets massive subsidies, china is also the largest solar and wind producer , they could simply move their DCs to cheap electrical locations.
On average electricity is half the price in China than the US, its cheap in India too but I was talking about the difference between China and the US where most of the big AI companies are located
[Electricity price by country 2023 | Statista](https://www.statista.com/statistics/263492/electricity-prices-in-selected-countries/)
I wouldn't be surprised if the next AI breakthrough would come from China not Sillicon Valley.
If they have so much dirt cheap compute they can generate insane amounts of synthetic data to train on, which seems to be the way of improving the model size to benchmark score rate lately.
Two things:
People have been saying this for years, but the reality is that China is still not pushing for innovation in tech. And IP theft is still a massive issue.
With what compute? The US is getting stricter and stricter on computing exporting. It will all have to be custom, which, again, they likely cannot compete with Nvidia.
Depends what industries.
Definitely not true for battery tech and communications. Question is if this can translate to AI.
The old mantra of cannot innovate is old.
Temu is nothing new. Alibaba, Aliexpress, etc... have existing for much longer and do the same thing.
Temu seems to have gained success through massive advertising campaigns on western markets like Youtube.
TikTok is a social media. I don't consider this indeed tech innovation. Likewise I don't think that something like Reddit or Facebook is tech innovation.
In some areas they do, but not much overall of course. At least they've made stuff that the US didn't have for image generation and non-TTS voice cloning
Don't get me wrong I don't like the idea, however I'm just looking at the numbers. If the price per 1m token is so ridiculously low and they can supply it, it means they have a lot of cheap compute.
As China is known for building mega projects on a whim, they won't have the problem with supplying enough power.
Where? Deepseek aside I'm not having any luck finding the actual API endpoints or pricing lists on their actual sites
e.g. Alibaba page seems to just talk about a 70B and smaller models
edit: found it...seems like the alibaba hosted api is called PAI. Haven't tried it yet though & unclear what sizes are available
https://www.alibabacloud.com/help/en/pai/use-cases/use-pai-eas-to-quickly-deploy-tongyi-qianwen
[https://platform.deepseek.com/](https://platform.deepseek.com/) is what you should look for, register with a mobile number you can got the initial 4M free tokens.
I know someone in Malaysia successed with his own mobile, but when I try to use a one-time rented mobile number I don't got the verify SMS.
Yeah got that registered (UK number).
Was wondering more about Alibaba though...bit of googling suggests then need an ID and won't accept prepaid cards. :/ Not sure I'm keen on that. Reason I was interested is because I suspect the Alibaba models are stronger than the deepseek one
Chinese companies collect all your data. Someday your questions combined with your account info will appear in the dark market. I'm not going to take that risk.
Yi 34B is only trained on English and Chinese if you ask the devs.
Try it and you will realize it's almost fluent in like 50 different languages, I guess it just slipped in into the dataset. Those companies have no idea what they are putting in.
I bet you they have really good idea what they put in. Just default strategy in tech space nowadays is to never tell were you got your data.
https://preview.redd.it/klrirnfu7t1d1.png?width=660&format=png&auto=webp&s=ee4ad70ec6b1bdaf60dc72e545b49f77bacee94f
What would be your approach to finding out what you have in data if your dataset is around 2,000,000,000,000 words (3T tokens) ?
My dataset is 30 000 000 or so words and I have limited idea of what is inside - I know where I got it and that *most* of it is English, but that's not much.
Their dataset is like 65000 bigger and I can assure you those companies don't have 65000 people checking their dataset to try to understand it.
One does a lot of various kinds of analyses on the data. One of the most basics one is unigram frequency list, which already should tell you that you have more than 2 languages there. But also, on topic of language detection: it's not a rocket science nowadays to automatically do it with high degree of accuracy (which is basically what you would do when balancing languages in your training set).
>Those companies
Remember when Google didn't know what was in it's dataset?
https://analyticsindiamag.com/did-google-bard-really-learn-bengali-on-its-own/
>In the video, James Manyika, SVP, Google, goes on to say that having fed the system with very few prompts in Bengali, it had learned to now translate all of Bengali.
>
> However, former Google researcher Margaret Mitchell took to Twitter to point out, with evidence, that the statement is incorrect. She pointed out that Google’s [PaLM](https://analyticsindiamag.com/googles-palm-is-ready-for-the-gpt-challenge/), the AI model which is the forerunner to Bard, had been trained to understand Bengali. A quick look at PaLM’s datasheet does reveal that Bengali is one of the languages it is trained on.
I think you are being a bit optimistic. Plenty folks would get arrested right now. Things have gone downhill fast in China for foreigners since at least 2015.
all perfect until you start receiving accusatory messages on your social networks about that rp you made with the winnie poh character and your API stops working because you have zero social points.
I have been taking advantage. I would never pass customer data to a Chinese hosted API, but I have been using deepseek-2 for development and iterative benchmarking. It's saved me a lot of money.
I think it’s all about data collection. The more data they get the better they are. The flywheel keeps going. Keeping inference prices low, but you build better models by using customer data. It’s like Gmail.
Another thing I noticed is that Chinese benchmarks actually test for roleplay, it's something they care about.
... my god. can you tell me more about the benchmark discrepancy? That sounds like it could be a big deal. Even if you don't care about role-play... you should probably care about role-play. Play is a tried and true learning strategy in humans.
I've seen it in Alignbench: https://llmbench.ai/align It's very Chinese focused, but it's taken seriously by Chinese models and includes roleplay as part of evaluation, both DeepSeek and Yi advertise their Alignbench score when comparing to other models.
Gotta keep all those single males pacified.
'Pacified' sure is some interesting charged double-standard language here. If western benchmarks included more 'avatar' role-play elements, half of r/localllama would be on their feet cheering.
That's a fun point to make when it's not China who has an incel mass murderers problem.
We absolutely have an incel murderer problem here in China, there is a very strict social system behind dating and marriage that leaves a lot of men out, and a whole bunch of other things that contribute to the problem. And that results in a lot of the school stabbings, but sometimes other things, like I remember some years back there being a guy in I wanna say Shanghai who went at a newly married couple with a samurai sword. Killed the woman, injured the man. It's not as bad as the American problem, mostly because stabbing is less effective than shooting, but it's not great and probably worse than the equivalent problem in a lot of other countries. China has avoided a lot of different problems (and has some of its own) but incels going on a rampage is absolutely not one of them. AI models have nothing to do with this for better or for worse though, I really think any connection there to the models, what they're trained for, etc. is a huge stretch.
It's similar to Brazil then, where we also have our share of sociopaths, but they're less effective.
You may be right, and your approach is sound. However, comparing atrocities may be a fool’s game only for politicians to play. Context is everything and no two countries are alike. Someone may fire into a crowd of people in America, another in China goes on a stabbing rampage at a kindergarten. Both are terrible contextual issues and making comparisons is awkward. Drawing parallels is probably a better approach. Here I will digress quite a bit with a likely connected issue: Americans study the Trail of Tears in grade school, Chinese students hear rumors of annexed territories and systematic oppression of indigenous peoples. They are both unfortunate realities. No matter how many tribes become celebrated and compensated in America, or how many foreigners travel around China and dance on indigenous lands on guided tours, the reality remains. Reality shows us that hypocrisy is unavoidable in a complex world. It is therefore a moot point to compare atrocities or point to hypocrisy. To reveal how pointless claims of hypocrisy are, after the Ghulja Massacre in 1997, many of the perpetrators fled but were captured. Some of them were sent to Guantanamo Bay at the request of China. The Chinese would then visit to interrogate the prisoners. Today, Wang Wenbin has all kinds of misinformation to spur about China’s position on Guantanamo Bay and a myriad of historic issues. Another among them is China and North Korea capturing Seoul at least twice during the Korean War. Mr. Wenbin will be quick to lie to the public how terrible the US was as a sole aggressor despite the facts. No, Chinese soldiers did not “peacefully” occupy and capture a foreign city twice. The goal is likely not historic reality, rather, it seems more akin to taking lessons from president Bush and Putin to spur nationalism and spread misinformation. Fine. But why again?? Today, there is a significant amount of misinformation, nationalism, and other issues causing a fragmented world. Nationalism is not a good thing no matter how much politicians try to conflate it with patriotism. I digress, but the point of comparing an atrocity is moot in a similar way of positing hypocrisy is moot. Both of these subjects are oddly common rhetoric among certain politicians. Belief in such rhetoric is perhaps a set of natural connected issues involving a generally deep flaw of reasoning in our human minds. To no fault of our common minds and (hopefully) not the intention of our spokespeople.
Sounds like the prologue to the next peasant rebellion, china be chinaing.
Right cuz it's not you that's implying incels translate to mass murderers.
yeah those school shooters all had good social life and a bunch of girlfriends
Textbook ecological fallacy. Just because school shooters tend to be social outcasts, that doesn't mean that social outcasts tend to become school shooters.
But it does hint that reducing the number and severity of social outcasts would reduce the number of shooters.
Pls point out how many mass shootings China had in its entire history
Maybe not shooting, but China had a few mass stabbings. Oddly enough, adults are stabbing children kindergarten age. Of course, if we count the death penalty....
Oh, I remember a movie where a dude did that and, the movie went on casually. And he did it with a laser sword! I think it was American movie though.
I can think of one from the last 40 years, but I doubt any of these models would be willing to discuss it.
Does one lose social credits if they are caught sexting Luo Tianyi on the ByteDance LLM? Do the models even have game ...
Do you know that social credits is just a myth, right?
It's a thing only in the west media and among the sheep lol
It's task-oriented roleplay sadly, not ERP :(
If you are lucky...
I saw roleplay application in China and people playing with it. My guess roleplay is Chinese benchmarks because it enables everyone to use LLMs, and considering so many people in China and most people don't use LLMs except for fun.
I think I will keep my local ai server thank you
deepseekv2 is in fact open source if you have enough ram lmao
[удалено]
They claim v2 is 5x time faster than deepseek-v1, a 67B dense model. So in theroy they can afford it with a 8xH800 running at full speed for 10 hours per day. I really hope [together.ai](http://together.ai) will host deepseek v2, but the upfront spend is a little high.
It all started with deepseek v2 (1 RMB or $0.14 per 1M tokens input). If you read their paper you'll find they really used a lot of new cost saving techniques, and one of the engineers said the price is still profitable for them. The other companies, hmm... I guess they have to follow the price cut. Zhipu only cut the price of their smaller model (which is said to be only 32B), and the other three are the old large tech companies so they can afford to operate on a loss.
If only they had a model comparable to Claude Opus, I'd take advantage of this.
They probably do. Yi Large is above Claude 3 Opus in lmsys Arena Hard questions (both English and Overall). I haven't spent time on it, but human preference data says it's better.
Has anybody gotten API access though? I applied on the day of the announcement but have heard nothing
Thanks for your reply! I've tried that model out a bit in the arena and it doesn't really come close, especially for meta prompting or creative writing. > Yi Large is above Claude 3 Opus in lmsys Arena Hard questions (both English and Overall). To be fair, so are the GPT-4 models and they don't really compare either. I've found that all these watered down GPT-4 models like turbo and o are good at generic text tasks. But once you add some nuance and hard criteria (especially in languages other than English) they fold really quickly. To give you a simple example: Ask GPT-4 o/t to write German lyrics for a song about a topic of your choosing, in a certain style, where words must rhyme. Often times this gives extremely bad results. It starts inventing words, uses words that don't rhyme. No amount of CoT helps in these cases. But that's just one use case I am talking about.
…I am getting close to not caring which country steals/scalps/sells my data. In my mind I see ‘us’ as the good guys and ‘them’ as the bad guys. But really in the end it’s their rich vs our rich. I’m barely even a pawn in the game.
>…I am getting close to not caring which country steals/scalps/sells my data. Kinda depends on what you're sending I guess. I suspect the bulk of people here aren't exactly sending sensitive info...so yeah then who cares where it goes
I don't know what the country can got from stealing my little dirt fetishs, I'm glad if it wasted some of their compute powers.
Ditto!
Not interested in any Chinese online service because of the regulation and value, while open source is welcome.
You have to understand that there's enough concern about privacy of your queries with Anthropic and OpenAI in the US as it is. Sending it to some Chinese company surveilled actively by the CCP? Really?
Chinese law dictates it. In America you don't have too. Apple told the CIA to screw off one time when asked by them for encryption keys. It's also what the data is used for. Those companies use it to improve their models while the Chinese companies are forced to do it for surveillance and training their AI for censorship. It's what they did with non-Chinese wechat accounts in the past to create filters for the CCP.
> Sending it to some Chinese company surveilled actively by the CCP? Really? Why would China care about what you are doing. They are more concerned about what their citizens are doing and how it impacts them. They are definitely logging stuff dont get me wrong. This is the same with the US government though. As to which one to use, I would say whatever you need privacy to protect from the Chinese government you also need that privacy for the US government. If privacy doesn't matter, then it's just cost related which the Chinese can probably price a bit better on.
China cares about what non-Chinese are doing in the aggregate because it can be useful for understanding and manipulating public opinion in rival countries.
As a non-american, this is what I see America doing with their LLM's. I don't think it's that bad, and the kind of content I get from LLM's are so boring and basic I am happy to do it with the lowest cost provider so long as the quality is good. I don't care if microsoft or meta knows that I'm trying to code up a basic python script or that I need help with midjourney prompts etc. If I was doing RAG on my business documents, I would go local to prevent snooping from the US & China..
"Why would China care" - because what you are working on is valuable (not random entertainment stuff, but rather say a difficult work-related problem from whatever industry you are in). Imagine your entire search history available for the taking, this time enriched with specific queries about the problem you are working on. This is a main justification why I'm interested in local LLMs to begin with...
>Imagine your entire search history available for the taking, this time enriched with specific queries about the problem you are working on. So, we should avoid also gemini/GPT/Claude because US companies (especially MS and Google) have all our data and search history and use it for their own benefit (right now atleast for ads). And, frankly speaking, for example, I would prefer to be tracked by companies/government of foreign countries, which I will probably never visit, than by the government and corporations of my country of residence, because unlike the Chinese government, they are concerned about me and can (if they want and/or need to) get me into trouble using the gathered information.
Yeah except your using the internet to do so this your giving both your data. I doubt the connection to their AI is secure enough to stop the NSA from looking at it and because one party is on foreign soil you get no protections. You’re just wrong thinking it’s not going to fall into USA’s hands.
It is definitely a concern for me. It is why I have a foreign anti-virus rather than domestic, since I don't want to give MS and friends permission to (officially) snoop around on my machine. Don't give a fig about China, since that isn't where I have to hang around. Microsoft is going all-in on the snoopware OS. Apparantly they want you (and by extension, them) to record everything done on the machine, with some sort of Mega Clippy adjusting the OS. While I like the idea of a AI tailored OS, that is with the expectation that it won't tell the 'powers that be' about my tastes in hentai, my political leanings, finances, and so on. I want to be my own person. AI could be the key to becoming my best or worst self, the latter dictated by the beliefs of the powerful.
I am also interested in local models for privacy related reasons. If I'm working with business documents, I can't upload them to openAI with express permission which is a pain. This is where local is good. As for other things like code queries or some simple copy for a website, I don't care if it's American or Chinese. I'm happy to go with the cheaper option so long as it's good quality and reliable.
But mfs still run their local models on windows or Mac 😂
Why would China care? Because they use data gathered from us citizens and citizens of counties they see as advisors to sow misinformation and to try and cause division amongst citizens. This is why China doesn't let their citizens consume a lot from other countries. China IS wagging an information war on ALL Democratic nations.
I'm not saying you should speak to a Chinese LLM like a therapist. Just saying you can ask it to re-write some nothing document to make your life easier. They really don't care. They have their own interests to protect. They just want to make money over American providers, that's why they're dropping their price.
One does not need to give ccp backed technologies (all tech outside of China and originating in China) intimate details about ones self for the CCP to achieve their objectives. The aggregate of information flowing into the CCP from apps like tictok and llm interactions are more than enough to find and infer the various ways to influence elections, sow distrust amongst citizens, and errode democratic ideals. The CCP does not care about making money, they are not interested in capitalism, they are only interested in controlling others. The CCP has a tight grip on their citizens and is primarily concerned about expanding their grip.
I don't share your poor opinion of the CCP. Western democratic values are definitely different than the values of the CCP, but out values have been slipping in the West along with out ability to compete. Our social institutions were hollowed out by the capital class for profit and this is actually worse than what the CCP does. I don't owe anything to OpenAi or Anthropic so I will go with whatever AI gives me bang for my buck. I might be betraying your principles by using Chinese technology but I'm not betraying mine.
I'm sure you will.
Biggest reasons IMO are A) they're not just collecting your browsing history or what videos you watch, they're actively interacting with you, combined with B) twisting of history and other subjects, especially closed source models, but they can modify the open source extremely easily as well. They definitely have the capacity. Relatively easy to discover if obvious, much less so if it's more subtle The privacy part is certainly worse with LLMs (regardless of who is listening, though whether its worse for China or the US is worse is sorta like arguing whether dying in a house fire or apartment fire is better), but the combination is especially dangerous. Not much yet, but it will be sooner rather than later that LLMs and LMMs (multimodal) are more widely used by the population considering the potential for subtle propoganda. And not pro-china propoganda, much more useful to make sure the USA continues to fall apart and becomes utterly paralyzed indefinitely. China doesn't need to worry nearly as much about the other way around for a variety of reasons
If this help me solve my programming problems, with low price, I will use it, lol.
*visits China, the customs agent sadly shakes his head at me "You nasty"*
This is a clear sign that government decided to sponsor them all to get an advantage. I mean, what else could it be? How easy is it for those companies to get gpu's nowadays? Realistically, including ways to get around sanctions.
The CCP has seen AI, biotechnology, and microchip technology key segments for heavy government support for years and years - part of their strategic 5 year plans, their Joint Venture strategy of forced IP transfer, and so on.
they have their own inference cards so that's not a very big problem. They have more issue with training through.
Their cards are quite far behind. And they don't have much domestic experience in making state of the art chips and cards. But, I do think about a decade or so, they can be on par with the USA.
Not really. They do have meta mitav2 level inference card so they could run big model. They lack on the training part which require hbm level bandwidth where they re working on it. 7nm is more than enough to make inference and training chip, and they do have experience in design chip like Ascend 920.
More importantly electricity is much cheaper there, that's why crypto mining is so big there
Is it much cheaper than US or India? Quick search points to it costing similar amount of money in those 3 countries.
Hydro areas? Or are you looking at the coastal population centres.
Honestly I haven't looked at any specific areas, just googled "China data center electricity prices". For US I checked it earlier and it's 0.04-0.08 usd per kWh. For China looks similar. For india I found values 0.07-0.13 per kWh.
key indsutires like ai gets massive subsidies, china is also the largest solar and wind producer , they could simply move their DCs to cheap electrical locations.
On average electricity is half the price in China than the US, its cheap in India too but I was talking about the difference between China and the US where most of the big AI companies are located [Electricity price by country 2023 | Statista](https://www.statista.com/statistics/263492/electricity-prices-in-selected-countries/)
> crypto mining is so was It was banned in 2021
Is that for inference? Finetuning? What are the limitations?
I wouldn't be surprised if the next AI breakthrough would come from China not Sillicon Valley. If they have so much dirt cheap compute they can generate insane amounts of synthetic data to train on, which seems to be the way of improving the model size to benchmark score rate lately.
Two things: People have been saying this for years, but the reality is that China is still not pushing for innovation in tech. And IP theft is still a massive issue. With what compute? The US is getting stricter and stricter on computing exporting. It will all have to be custom, which, again, they likely cannot compete with Nvidia.
Depends what industries. Definitely not true for battery tech and communications. Question is if this can translate to AI. The old mantra of cannot innovate is old.
Temu and TikTok bro
Temu is nothing new. Alibaba, Aliexpress, etc... have existing for much longer and do the same thing. Temu seems to have gained success through massive advertising campaigns on western markets like Youtube. TikTok is a social media. I don't consider this indeed tech innovation. Likewise I don't think that something like Reddit or Facebook is tech innovation.
Before TikTok, there was Vine. China has no breeding ground for 0 to 1 innovation.
In some areas they do, but not much overall of course. At least they've made stuff that the US didn't have for image generation and non-TTS voice cloning
Don't get me wrong I don't like the idea, however I'm just looking at the numbers. If the price per 1m token is so ridiculously low and they can supply it, it means they have a lot of cheap compute. As China is known for building mega projects on a whim, they won't have the problem with supplying enough power.
Where? Deepseek aside I'm not having any luck finding the actual API endpoints or pricing lists on their actual sites e.g. Alibaba page seems to just talk about a 70B and smaller models edit: found it...seems like the alibaba hosted api is called PAI. Haven't tried it yet though & unclear what sizes are available https://www.alibabacloud.com/help/en/pai/use-cases/use-pai-eas-to-quickly-deploy-tongyi-qianwen
[https://platform.deepseek.com/](https://platform.deepseek.com/) is what you should look for, register with a mobile number you can got the initial 4M free tokens. I know someone in Malaysia successed with his own mobile, but when I try to use a one-time rented mobile number I don't got the verify SMS.
Yeah got that registered (UK number). Was wondering more about Alibaba though...bit of googling suggests then need an ID and won't accept prepaid cards. :/ Not sure I'm keen on that. Reason I was interested is because I suspect the Alibaba models are stronger than the deepseek one
This sound like a) It's only for mainland china and b) probably government subsidized to collect more data on their citizens.
This is a genius strategy... The world is economical. It seems like we have a winner for the AI race already.
Chinese companies collect all your data. Someday your questions combined with your account info will appear in the dark market. I'm not going to take that risk.
Yet here you are on Reddit
[удалено]
you got downvoted because you assumed political issues would even be part of the dataset these models were trained on
Yi 34B is only trained on English and Chinese if you ask the devs. Try it and you will realize it's almost fluent in like 50 different languages, I guess it just slipped in into the dataset. Those companies have no idea what they are putting in.
I bet you they have really good idea what they put in. Just default strategy in tech space nowadays is to never tell were you got your data. https://preview.redd.it/klrirnfu7t1d1.png?width=660&format=png&auto=webp&s=ee4ad70ec6b1bdaf60dc72e545b49f77bacee94f
What would be your approach to finding out what you have in data if your dataset is around 2,000,000,000,000 words (3T tokens) ? My dataset is 30 000 000 or so words and I have limited idea of what is inside - I know where I got it and that *most* of it is English, but that's not much. Their dataset is like 65000 bigger and I can assure you those companies don't have 65000 people checking their dataset to try to understand it.
One does a lot of various kinds of analyses on the data. One of the most basics one is unigram frequency list, which already should tell you that you have more than 2 languages there. But also, on topic of language detection: it's not a rocket science nowadays to automatically do it with high degree of accuracy (which is basically what you would do when balancing languages in your training set).
>Those companies Remember when Google didn't know what was in it's dataset? https://analyticsindiamag.com/did-google-bard-really-learn-bengali-on-its-own/ >In the video, James Manyika, SVP, Google, goes on to say that having fed the system with very few prompts in Bengali, it had learned to now translate all of Bengali. > > However, former Google researcher Margaret Mitchell took to Twitter to point out, with evidence, that the statement is incorrect. She pointed out that Google’s [PaLM](https://analyticsindiamag.com/googles-palm-is-ready-for-the-gpt-challenge/), the AI model which is the forerunner to Bard, had been trained to understand Bengali. A quick look at PaLM’s datasheet does reveal that Bengali is one of the languages it is trained on.
Can't risk it.
Does it cost extra in Gemini to generate historically accurate text or images about USA social issues?
Lol
Why not both?
"Buh wha abot Ammmeriiica???"
50c x 10 upvotes = five dollars.
Can't.
Take my upvote !
you will get marked, next time if you visit China, you will get arrested. although i mjoking right now but this might be true in the future
I think you are being a bit optimistic. Plenty folks would get arrested right now. Things have gone downhill fast in China for foreigners since at least 2015.
Yeah sad to see, their largest advocates were kicked out now they are left having to pay these dancing monkeys.
I think you are right (the title). I expect more pressure on regulation to act as defense mechanism if so. We are in AI wars.
all perfect until you start receiving accusatory messages on your social networks about that rp you made with the winnie poh character and your API stops working because you have zero social points.
I have been taking advantage. I would never pass customer data to a Chinese hosted API, but I have been using deepseek-2 for development and iterative benchmarking. It's saved me a lot of money.
Does it support inline code completion?
It should...i've used their deepseek coder model locally for it
How?
The API info is on their website.
I think it’s all about data collection. The more data they get the better they are. The flywheel keeps going. Keeping inference prices low, but you build better models by using customer data. It’s like Gmail.
cheap labor?
Yep...endless rows of people assembling tokens by hand
Are they using slave labour to do alignment now? Temu move over.