Newsletter
CoinSharks
  • Bitcoin
  • Ethereum
  • Altcoins
  • Nasdaq
  • NYSE
  • Regulation
  • Shark School
  • Premium Services
No Result
View All Result
  • Bitcoin
  • Ethereum
  • Altcoins
  • Nasdaq
  • NYSE
  • Regulation
  • Shark School
  • Premium Services
No Result
View All Result
CoinSharks
No Result
View All Result
Home News

Meta Introduces Megabyte System, Addressing Major Roadblocks for GPTs

The Megalodon by The Megalodon
May 26, 2023
in News, Bitcoin
0
Meta’s positive first quarter overshadowed by $4B loss in Metaverse division
390
SHARES
1.8k
VIEWS
Share on FacebookShare on Twitter

Researchers at Meta AI have made a significant breakthrough in tackling the tokenization challenge associated with GPT (Generative Pre-trained Transformer) models.

In their recent pre-print research, Meta AI introduces the innovative Megabyte framework for constructing GPT systems. This architecture aims to process large-scale datasets, including images, novels, and videos, without relying on tokenization, a lossy compression process.

Related articles

US district judge refers FTX independent examiner issue to appellate court

US district judge refers FTX independent examiner issue to appellate court

May 31, 2023
Class-Action Lawsuit Filed Against Bancor DAO for Alleged Failure to Deliver on Impermanent Loss Protection Promises

U.S. prosecutors state that if the Bahamas raises objections, certain SBF charges will be dropped

May 31, 2023

Tokenization, which converts bytes to tokens for processing, enables AI systems to handle data as numerical sequences. For example, the sentence “my favorite color is red” would be transformed into the token string “3666, 4004, 3124, 318, 2266, 13” for processing by OpenAI’s ChatGPT.

However, the current state-of-the-art systems still face limitations even with tokenization. GPT-3.5 can process slightly over 4,000 tokens, while GPT-4 has a limit of around 32,000 tokens.

Megabyte, on the other hand, introduces a novel multi-layer prediction architecture that operates in an end-to-end manner, enabling modeling of more than 1 million bytes of data. This revolutionary approach eliminates the need for tokenization, providing a 3,025% increase in text capacity compared to GPT-4. Megabyte can process text documents with approximately 750,000 words, equivalent to Leo Tolstoy’s War and Peace and two additional average-length novels.

The Megabyte model exhibits excellent performance in ImageNet tests and audio file processing benchmarks, often outperforming existing byte-based transformer models like DeepMind’s Perciever AR. It matches PerceiverAR’s state-of-the-art performance while utilizing only half the computational resources.

This research has profound implications, as overcoming tokenization barriers enables training AI models with enhanced support for non-English languages and facilitates the democratization of these technologies worldwide. It empowers the development of cryptocurrency trading bots, decentralized autonomous organizations, and other applications using native language codes.

Moreover, Megabyte expands the capabilities of models like ChatGPT to handle multimedia files, such as images, videos, and audio, with similar time and energy consumption as text processing, opening up new possibilities for AI applications.

Share156Tweet98

Related Posts

US district judge refers FTX independent examiner issue to appellate court

US district judge refers FTX independent examiner issue to appellate court

by The Megalodon
May 31, 2023

A motion for the appointment of an independent examiner in the bankruptcy case of crypto exchange FTX has been referred...

Class-Action Lawsuit Filed Against Bancor DAO for Alleged Failure to Deliver on Impermanent Loss Protection Promises

U.S. prosecutors state that if the Bahamas raises objections, certain SBF charges will be dropped

by The Megalodon
May 31, 2023

According to a filing in the U.S. District Court for the Southern District of New York, some of the charges...

Bybit cryptocurrency exchange withdraws from Canada due to ‘recent regulatory changes’

Bybit cryptocurrency exchange withdraws from Canada due to ‘recent regulatory changes’

by The Megalodon
May 31, 2023

Dubai-based cryptocurrency exchange Bybit has revealed its decision to halt operations in Canada due to recent regulatory developments. The exchange...

AI Experts Issue Document Comparing the Risk of ‘Extinction from AI’ to Pandemics and Nuclear War

AI Experts Issue Document Comparing the Risk of ‘Extinction from AI’ to Pandemics and Nuclear War

by The Megalodon
May 31, 2023

An open statement published by the Center for AI Safety (CAIS) has garnered signatures from numerous AI experts, including the...

Competing regulatory bills take center stage in US House stablecoin hearing

US CFTC Releases Letter Addressing Compliance in Three Areas of Digital Asset Derivatives and Clearing

by The Megalodon
May 31, 2023

The United States Commodity Futures Trading Commission (CFTC) has issued a staff advisory letter to registered derivatives clearing organizations (DCOs)...

Load More
  • Trending
  • Comments
  • Latest
President Biden Proposes Phased-In 30% Tax on Cryptocurrency Mining Electricity Usage

President Biden Proposes Phased-In 30% Tax on Cryptocurrency Mining Electricity Usage

March 15, 2023
Kaspersky reports a 40% increase in crypto phishing attacks within a year

Kaspersky reports a 40% increase in crypto phishing attacks within a year

April 17, 2023
Breaking Barriers: Women in Web3 Leading the Charge for Diversity and Inclusion

Breaking Barriers: Women in Web3 Leading the Charge for Diversity and Inclusion

March 15, 2023
Goldman Sachs Identifies Two High-Yield Dividend Stocks Yielding Up to 10%

Goldman Sachs Identifies Two High-Yield Dividend Stocks Yielding Up to 10%

April 24, 2023
Protecting Your Crypto: The Importance of AML Frameworks in Web3

Protecting Your Crypto: The Importance of AML Frameworks in Web3

President Biden Proposes Phased-In 30% Tax on Cryptocurrency Mining Electricity Usage

President Biden Proposes Phased-In 30% Tax on Cryptocurrency Mining Electricity Usage

Explaining the Reasons for the Recent Downturn in Ethereum (ETH) Price: A Macro Analysis

Explaining the Reasons for the Recent Downturn in Ethereum (ETH) Price: A Macro Analysis

Breaking Barriers: Women in Web3 Leading the Charge for Diversity and Inclusion

Breaking Barriers: Women in Web3 Leading the Charge for Diversity and Inclusion

US district judge refers FTX independent examiner issue to appellate court

US district judge refers FTX independent examiner issue to appellate court

May 31, 2023
Class-Action Lawsuit Filed Against Bancor DAO for Alleged Failure to Deliver on Impermanent Loss Protection Promises

U.S. prosecutors state that if the Bahamas raises objections, certain SBF charges will be dropped

May 31, 2023
Bybit cryptocurrency exchange withdraws from Canada due to ‘recent regulatory changes’

Bybit cryptocurrency exchange withdraws from Canada due to ‘recent regulatory changes’

May 31, 2023
AI Experts Issue Document Comparing the Risk of ‘Extinction from AI’ to Pandemics and Nuclear War

AI Experts Issue Document Comparing the Risk of ‘Extinction from AI’ to Pandemics and Nuclear War

May 31, 2023
  • Premium Services
  • Terms of Service
  • Privacy Policy

All Rights Reserved. CoinSharks.

No Result
View All Result
  • Bitcoin
  • Ethereum
  • Altcoins
  • Nasdaq
  • NYSE
  • Regulation
  • Shark School
  • Premium Services

All Rights Reserved. CoinSharks.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy and Cookie Policy.