OpenAI has delivered a series of impressive advances in AI that works with language in recent years by taking existing machine-learning algorithms and scaling them up to previously unimagined size. GPT-4, the latest of those projects, was likely trained using trillions of words of text and many thousands of powerful computer chips. The process cost over $100 million.
But the company’s CEO, Sam Altman, says further progress will not come from making models bigger. “I think we’re at the end of the era where it’s going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We’ll make them better in other ways.”
Altman’s declaration suggests an unexpected twist in the race to develop and deploy new AI algorithms. Since OpenAI launched ChatGPT in November, Microsoft has used the underlying technology to add a chatbot to its Bing search engine, and Google has launched a rival chatbot called Bard. Many people have rushed to experiment with using the new breed of chatbot to help with work or personal tasks.
Meanwhile, numerous well-funded startups, including Anthropic, AI21, Cohere, and Character.AI, are throwing enormous resources into building ever larger algorithms in an effort to catch up with OpenAI’s technology. The initial version of ChatGPT was based on a slightly upgraded version of GPT-3, but users can now also access a version powered by the more capable GPT-4.
Altman’s statement suggests that GPT-4 could be the last major advance to emerge from OpenAI’s strategy of making the models bigger and feeding them more data. He did not say what kind of research strategies or techniques might take its place. In the paper describing GPT-4, OpenAI says its estimates suggest diminishing returns on scaling up model size. Altman said there are also physical limits to how many data centers the company can build and how quickly it can build them.
Nick Frosst, a cofounder at Cohere who previously worked on AI at Google, says Altman’s feeling that going bigger will not work indefinitely rings true. He, too, believes that progress on transformers, the type of machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling. “There are lots of ways of making transformers way, way better and more useful, and lots of them don’t involve adding parameters to the model,” he says. Frosst says that new AI model designs, or architectures, and further tuning based on human feedback are promising directions that many researchers are already exploring.
Each version of OpenAI’s influential family of language algorithms consists of an artificial neural network, software loosely inspired by the way neurons work together, which is trained to predict the words that should follow a given string of text.
The first of these language models, GPT-2, was announced in 2019. In its largest form, it had 1.5 billion parameters, a measure of the number of adjustable connections between its crude artificial neurons.
At the time, that was extremely large compared to previous systems, thanks in part to OpenAI researchers finding that scaling up made the model more coherent. And the company made GPT-2’s successor, GPT-3, announced in 2020, still bigger, with a whopping 175 billion parameters. That system’s broad abilities to generate poems, emails, and other text helped convince other companies and research institutions to push their own AI models to similar and even greater size.
After ChatGPT debuted in November, meme makers and tech pundits speculated that GPT-4, when it arrived, would be a model of vertigo-inducing size and complexity. Yet when OpenAI finally announced the new artificial intelligence model, the company didn’t disclose how big it is—perhaps because size is no longer all that matters. At the MIT event, Altman was asked if training GPT-4 cost $100 million; he replied, “It’s more than that.”
Although OpenAI is keeping GPT-4’s size and inner workings secret, it is likely that some of its intelligence already comes from looking beyond just scale. On possibility is that it used a method called reinforcement learning with human feedback, which was used to enhance ChatGPT. It involves having humans judge the quality of the model’s answers to steer it towards providing responses more likely to be judged as high quality.
The remarkable capabilities of GPT-4 have stunned some experts and sparked debate over the potential for AI to transform the economy but also spread disinformation and eliminate jobs. Some AI experts, tech entrepreneurs including Elon Musk, and scientists recently wrote an open letter calling for a six-month pause on the development of anything more powerful than GPT-4.
At MIT last week, Altman confirmed that his company is not currently developing GPT-5. “An earlier version of the letter claimed OpenAI is training GPT-5 right now,” he said. “We are not, and won’t for some time.”
You May also like = TriphalaRas Benefits
The post THE STUNNING CAPABILITIES of ChatGPT appeared first on VVDesigns.]]>Baruchel: There’s so much buzz around generative AI right now that it’s hard not to feel skeptical about some of its applications. Why are you so interested in its potential in the context of access to justice in India?
Malhan: There are more than 1.4 billion people living in India and only about 10 percent of the population can access justice because it’s much too costly for the average person. AI has the potential to absolutely crush the cost of transaction and level the playing field by helping people understand things like what their rights are; what to look for if and when they need a lawyer; or what legal questions to ask. AI could also help lawyers and individuals identify whether a property deed is up to standard. It can cut down research time and help unclog court dockets. If we can drop some of those costs to next to zero it can lead to a massive explosion in access to justice in countries where the system is hugely underfunded, whether it’s in South East Asia or Africa.
But for that, we need publicly minded innovators to build the middle layer of AI for Justice, and then a bunch of entrepreneurs to build solutions that serve people from all walks of life. Most people in our space will create AI to help large companies navigate litigation, handle documents, and generally serve the well-paying class. There is no doubt we’re about to see an incredible wave of innovation, but is it going to be affordable? Is it going to be directed towards public ends?
Hanae Baruchel: What has this rapid evolution in generative AI meant for organizations like yours?
Sachin Malhan: For our own work developing an ecosystem of AI for Justice solutions in India, the potential is revolutionary. We used to spend hundreds of hours teaching the computer how to recognize and structure different types of data. For example, with one of our OpenNyAI apps –in Hindi “nyay” means justice– we wanted the computer to recognize what a court judgment looks like and highlight the key facts to create judgment summaries. This meant we had to annotate 700 to 750 court records ourselves before it could start understanding the patterns. This is lengthy, painstaking and expensive work. With the sophistication of GPT, LaMDA and other large language models, you could now dump 500,000 judgements or even a million all at once and it would do the annotating practically on its own, “unsupervised.”
Baruchel: You have already started incorporating generative AI into your work. Can you give an example?
Malhan: Yes. We are in the middle of a small pilot called Jugalbandi, where we are training ChatGPT to answer any question pertaining to government entitlements in India, like eligibility for an affordable housing scheme. We’re feeding in the government scheme information – the clauses, the eligibility criteria, etc. – to ensure accuracy and explainability, and ChatGPT adds an interactive layer on top of it.
Baruchel: You mean I could go into your app and say: “I’m in Bombay. Can you help me?”
Malhan: Exactly, and the system would answer: “What kind of support are you looking for? Would housing be of interest?” And you might say “Oh, yeah, housing would be great.” It will start asking things like “How old are you? Do you have an existing house? Do you have dependents?” It will interact with you at your own level of conversational comfort.
The key here is that it will work even if you are semi-literate or illiterate, in your own local language because we’re integrating Bhashini ULCA, an open-source data project that enables voice recognition and translation from a dozen or so Indian languages to another. So I could ask ChatGPT a question in Hindi or Bengali and it would respond to me both by text and through a voice message in my own language. For the first time ever, someone in a remote village in India will be able to ask questions and get answers immediately about what government entitlements they might be eligible for. This is a potential gamechanger because most of the research shows that last mile access to essential services fails because people don’t know what is available to them or how to use existing systems.
Baruchel: How do you factor in the risks of applying AI in such high-stake situations? When you talk about government entitlements and social welfare, we’re basically talking about the most vulnerable segments of society.
Malhan: Things are moving so fast right now that this is a real and legitimate concern. Most people aren’t taking the time to consider questions of fair use or privacy even. This is why it has been so important for us to build this middle layer of AI applications as a collaborative, open source effort. Someone is going to build these tools whether we do it or not, but if we manage to build it as part of a community effort with a truly diverse group of people who are impact oriented and can offer perspectives on the things to watch out for we’ll be much better equipped to mitigate unintended consequences.
Baruchel: What is missing for more people to build out technology in this way?
Malhan: We need to create the spaces where entrepreneurs, innovators and academics who are interested in building better AI and better AI applications can think about the hard questions together. In India we’re working with a wide range of technologists, grassroot organizations and lawyers, to catch issues as they arise and design this middle layer of AI for Justice in a way that works for everyone. We need to build a global Justice AI entrepreneur ecosystem to develop the parameters for conversational AI privacy rules, conversational AI bias, and more. Things are moving so fast that we don’t even have time to anticipate the problems. That is why when Sam Altman, CEO of OpenAI, was asked “What do you think we’re not talking about?” he surprised a lot of people when he said, “Universal Basic Income.”
The post Will ChatGPT Revolutionize Access To Justice In India? appeared first on VVDesigns.]]>