macOS vs Windows
iOS vs Android
OpenAI vs Meta AI?
Edit: Another observation is Meta seems to open things defensively. Facebook had a rich developer platform until Facebook stopped needing to grab market share. Meta’s VR platform was closed until Apple challenged them with Vision Pro. Then Meta announced open sourcing Horizon OS. I wonder if Meta will truly keep things open if they win, or if it’s more like a case of EEE?
And the OS being open isn't really that relevant unless you're a system integrator. I really doubt whether this is in response to Apple. They seem to operate in entirely different markets. I love VR but I can never buy a vision pro. It's twice what I've ever spent on my most expensive car.
Facebook the site had a rich platform until they had to secure it against people stealing your data, and your mate's data and pretending that they had a really rich dataset on most of the western world's adult population (ie Cambridge analytica )
Horizon OS is a fucking mess and isn't anything really to do with Apple, its more to do with the making sure samsung don't use android for what ever shit they produce.
OpenAI aren't going to compete with Meta, as they aren't in the same game. OpenAI has to make money from its offerings, Meta's AI shit is a byproduct of other things (see massive spending on "reality labs")
1. Spend billions on a product, then give it away for free.
2. ???
3. Profit!
> Then Meta announced open sourcing Horizon OS.
Open sourcing isn't really the right term, they're allowing third party hardware vendors to use it but it's still proprietary. Horizon OS is built on top of Android and they're following the Android playbook where the core is technically open source but the version nearly everyone actually uses has a bunch of proprietary Google (or Meta) software layered on top, and Google (or Meta) dictates the terms of using that software, which lets them ensure that revenue always flows back to Google (or Meta) regardless of who made the hardware.
1. Meta showed off automatic audio translation that preserves speaker's voices. Content creators can now expand their following beyond their spoken languages, generating more ad impressions.
1. Spend billions on a product.
2. Make it free to work with and charge to commercialize it.
3. Profit
Said no one ever.
Probably LLM backed products. Likely marketing tools if anything.
But not the same way OAI sells LLMs
Eh, 'OpenAI' is a swipe at OpenAI.
Famously the Microsoft strategy against Java.
https://en.m.wikipedia.org/wiki/Embrace,_extend,_and_extingu...
I think the bet is that many other things will use their llama that you then use. Perhaps without you even knowing this.
A simple narrative doesn't exist connecting these three product battles. The only thing that comes to mind is macOS, iOS and OpenAI are designed for the novice. You could probably add Coinbase vs Binance or Uber vs Lyft or Facebook vs Google Plus and be able to keep that narrative.
I'm kind of split about this. Yes, Facebook done a lot of great Open Source in the past, and I'm sure they'll do more great Open Source in the future.
But it's really hard to see them in a positive light when they keep misleading people about Llama, and publish blog posts that say how important Open Source is etc etc, then refuse to actually release Llama as Open Source, refuse to elaborate on why they see it as Open Source while no one else does it and refuse to take a step back and understand how the FOSS community feels when they actively mislead people like this.
The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work.
Where "open source" is misleading with Llama is that it's restricted to companies under a certain size and has restrictions for what you can and can't do with it. That kind of restriction undermines the freedoms promised by the phrase "open source", and it's concerning to me that people have gotten so fixating on weights vs data when there's a big gap in the freedoms offered on the weights.
For everyone, besides OpenAI etc.
I don't think most people expected that to happen so quickly or frankly at all.
It might not all be open-source, and they are doing it with an expectation of long-term profit, but they are earnestly pushing the horizons (pun intended) of the field and taking-on lots of risk for everyone else.
It's undeniable now that they are a serious and innovative engineering organization, while Google is rapidly loosing that reputation.
Most new products fail at meta, because they become a "priority", throw thousands of engineers at the problem and get bogged down in managing a massive oversubscription of engineers to useful work ratio.
Threads happened because a few people managed to convince each other to take a risk and build a instagram/mastodon chimera. They managed to show enough progress to continue without getting fucked in the performance review, but not enough for an exec to get excited about building an empire around it.
So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra).
Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up:
> "How much has Meta's valuation increased since they released their first open source model"
Answer (edited):
> Closing price on February 23, 2023: $509.50 > Closing price on October 11, 2024: $573.68 > The increase in stock price is $64.18 per share. > Total increase = Price increase per share × Number of outstanding shares > Total increase = $64.18 × 2,534,000,000 = $162,632,100,000 > Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023.
They seem to be making the right choices!
Tough to tell, given nobody is turning a net profit on LLMs yet.
Companies have a tendency to develop neuroses, though, just like people. Apple’s near miss with bankruptcy fuelled cash hoarding. For Facebook, their disastrous IPO and near miss of mobile seems to have made them hyper aware of the Innovator’s Dilemma. $400mm spent on a defensive move is certainly wider than tens of billions on the metaverse.
Perhaps, but Meta is definitely getting some money back from ad impressions supported by AI-generated content.
What’s unclear is if this is a short-term revenue benefit from people fucking around with the newest, shiniest model, or recurring revenue that is only appearing unprofitable because the frontier is advancing so quickly.
I know nothing about the enterprise side of OpenAI but I'm sure they're profitable there. I doubt the subscription cost of a single power user of ChatGPT Plus covers the water they consume as a single user (This is probably an exaggeration, but I think I'm in the ballpark).
I suspect in the case of Meta and other big players, profit isn't necessary required to bring substantial value. Imagine their model being able to help them moderate more fairly and accurately. This alone could prevent potential legal actions from individuals, companies, and governments.
They’re private companies. If they can’t tie it to profit, it’s not adding value.
> being able to help them moderate more fairly and accurately
This reduces legal costs and increases strategic flexibility. Sort of like HR or legal departments: cost centres add value by controlling costs, a critical component of profitability.
Not true, even strictly from an accounting perspective. If you spend $400M and build an asset that is worth >$400M, then you have increased the value of the company without modifying profit. For example, if a company were to buy land, and build a building, that building has a value regardless of if it is associated with any revenue.
Why does it have value? It’s because it can be rented, occupied for productive use or sold to someone who can do either of those. Revenue-free assets are essentially money. (Companies aren’t in the business of non-revenue non-monetary assets—that’s the domain of society at large.)
Smart companies understand other types of value exist.
If a democratic population hates you, it is harder to convince politicians to do your bidding. (Not impossible, just harder!)
If potential employees don't think kindly of you, it is harder to recruit.
Llama is a constant source of good PR for Meta in the developer community. Compared to just a couple of years ago when they were mostly laughed at by devs for metaverse stuff. Now it is "holy cow Zuck is standing up to Microsoft and Amazon and democratizing AI!"
With Llama, Meta has got great PR, and also developed cutting edge tech.
They also get to benefit from thousands of developers trying to make Meta's models run more efficiently.
These numbers are totally wrong, and it takes about 30 seconds to look it up. It closed at 589.95 on 2024-10-11 and 172.04 on 2023-02-23. The other numbers appear to be wrong too.
All thanks to their investments in VR! Wait guys where are you going
Big tech is getting increasing scrutiny by regulators, etc for monopolistic/anticompetitive practices/etc[0]. From S&P:
"Regulatory scrutiny is likely to translate into fewer mega deals, however. Adobe Inc. recently abandoned its $20 billion acquisition of Figma Inc. in the face of ongoing opposition from the UK's Competition and Markets Authority and the EU's European Commission. While Microsoft Corp. was able to close its $68.7 billion Activision Blizzard buy, it took nearly two years to complete the deal amid an intense fight with the regulators.
The harsh regulatory environment will continue to have a chilling effect on large tech M&A, but strategic buyers are likely to focus on smaller tuck-ins that do not tempt regulators to intervene."
So instead of growing further by spend on acquiring/rolling out new platforms to draw further scrutiny, they're shoveling piles of cash towards other capital expenses (with accounting tricks) that don't get further attention/scrutiny from regulators.
Meta doesn't use this infrastructure just for LLaMA - they get to use these hundreds of millions of dollars of spend to extract more value (deeper, not wider) from the users and platforms they already have.
Case in point: AI generated ads[1]. There's a not-too-distant future where the ads displayed on Facebook are an AI generated video of you in a new Toyota (or whatever). In terms of even open models Meta is much more than LLaMA. They have done a lot with speech (seamless), vision (segment anything), and much more.
That model training, inference, etc is running on these Nvidia GPUs as well.
There is also a take from Reid Hoffman saying that buying 100,000 GPUs or whatever is basically "table stakes" at this point[2]. Boards, analysts, the market, etc says "What I know for sure is this AI stuff needs Nvidia GPUs. Elon has 100k. How many do we have?".
So it's also a Cold War arms race/missile gap scenario. Plus, don't discount ego/"male anatomy measuring contest" that goes on with these guys. Consider this story where a bunch of centi-billionaires got together at Nobu to fight over Nvidia GPU shipments[3].
[0] - https://www.spglobal.com/marketintelligence/en/news-insights...
[1] - https://www.theverge.com/2024/10/8/24265065/meta-ai-edited-v...
[2] - https://www.theinformation.com/articles/openai-investor-hoff...
[3] - https://finance.yahoo.com/news/elon-musk-oracles-larry-ellis...
They don't need to make money, increase their value. They need to ward off existential risk. The best way to insure they aren't locked out from the future is to keep the future open. That's the only hope they have to maintain the locks they already have on much of the information technology world.
They’re all blowing billions of dollars on NVIDIA hardware with like 70% margin and with triton backing PyTorch it shouldn’t be that hard to move off of CUDA stack.
https://azure.microsoft.com/en-us/blog/azure-maia-for-the-er...
They're not teaming up and neither design is open, but they definitely have their own designs.
I think if the tooling and supply chain falls into place it would surprise me if meta and friends didn't make their own chips, assuming it was a good fit of course.
Note: AMD's missed opportunity here is so bad people jump "make their own chip" rather than "buy AMD". Although watch that space.
For a small fraction of that they could poach a ton of people from NVIDIA and publish a new open chip spec that anyone could manufacture.
https://www.fool.com/investing/2024/09/12/46-nvidias-30-bill...
Apple's already not using nVidia chips to train its models.
Quoting Yann LeCun (Vice-President, Chief AI Scientist at Meta):
AI datacenters will be built next to energy production sites that can produce
gigawatt-scale, low-cost, low-emission electricity continuously.
Basically, next to nuclear power plants.
The advantage is that there is no need for expensive and wasteful
long-distance distribution infrastructure.
Note: Yes, solar and wind are nice and all, but they require lots of land
and massive-scale energy storage systems for when there is too little sun
and/or wind. Neither simple nor cheap.
https://x.com/ylecun/status/1837875035270263014It’s good that a large company like Meta seems to be pushing it forward with openllama 3.2.
Open, quality LLMs must lead to some tense discussions at OpenAI and Anthropic about whether they should be open or closed.
OpenAI Whisper is open source and it’s extremely good although it appeared to me the online version is much faster.
It's not that Meta has anything against moats, it's just that they've seen how it works when they try to build their moat on top of someone else's (i.e. Apple and Google re: mobile) and it didn't work out too well.
So being committed to an open, standards-based strategy is an easy way for them to avoid most of the risks.
[0] https://engineering.fb.com/2024/10/15/data-infrastructure/op...
[1] https://ai.meta.com/blog/next-generation-meta-training-infer...
Would be far more interesting to see MTIA in an edge compute PCIe form.
See, e.g. [0] for an example of Facebook (now: Meta) for admirable dissemination of knowledge.
[0] https://engineering.fb.com/2015/06/26/security/fighting-spam...
Facebooks open efforts here are completely great, IMHO, and have benefited me personally and professionally.
Or maybe Open as in “open for business”.