OpenAI Codex: Difference between revisions
m →Issues: forgot one |
→Capabilities: +source, number |
||
| Line 6: | Line 6: | ||
== Capabilities == |
== Capabilities == |
||
Based on GPT-3, a [[neural network]] trained on text, Codex has additionally been trained on |
Based on GPT-3, a [[neural network]] trained on text, Codex has additionally been trained on 159 gigabytes of [[Python (programming language)|Python]] code from 54 million [[GitHub]] repositories.<ref name="VB-bias">{{Cite news|last=Wiggers|first=Kyle|date=July 8, 2021|title=OpenAI warns AI behind GitHub’s Copilot may be susceptible to bias|work=[[VentureBeat]]|url=https://venturebeat.com/2021/07/08/openai-warns-ai-behind-githubs-copilot-may-be-susceptible-to-bias/|access-date=2021-09-03}}</ref><ref name="IQ">{{Cite news|last=Alford|first=Anthony|date=August 31, 2021|title=OpenAI Announces 12 Billion Parameter Code-Generation AI Codex|work=InfoQ|url=https://www.infoq.com/news/2021/08/openai-codex/|access-date=2021-09-03}}</ref> OpenAI has stated that Codex can complete approximately 37% of requests and is meant to make human programming faster rather than replace it; according to OpenAI's blog, Codex excels most at "mapping [...] simple problems to existing code", which they describe as "probably the least fun part of programming".<ref name="SH">{{Cite news|last=Dorrier|first=Jason|date=August 15, 2021|title=OpenAI’s Codex Translates Everyday Language Into Computer Code|work=[[SingularityHub]]|url=https://singularityhub.com/2021/08/15/openais-codex-translates-everyday-language-into-computer-code/|access-date=2021-09-03}}</ref><ref name="VB">{{Cite news|last=Dickson|first=Ben|date=August 16, 2021|title=What to expect from OpenAI’s Codex API|work=[[VentureBeat]]|url=https://venturebeat.com/2021/08/16/what-to-expect-from-openais-codex-api/|access-date=2021-09-03}}</ref> According to a paper written by OpenAI researchers, when attempting each test case 100 times, 70.2% of prompts had working solutions.<ref name="arXiv">{{Cite arxiv|last=Chen|first=Mark|last2=Tworek|first2=Jerry|last3=Jun|first3=Heewoo|last4=Yuan|first4=Qiming|last5=Pinto|first5=Henrique Ponde de Oliveira|last6=Kaplan|first6=Jared|last7=Edwards|first7=Harri|last8=Burda|first8=Yuri|last9=Joseph|first9=Nicholas|last10=Brockman|first10=Greg|last11=Ray|first11=Alex|date=2021-07-14|title=Evaluating Large Language Models Trained on Code |arxiv=2107.03374 |class=cs}}</ref> |
||
OpenAI claims that Codex is able to function in over a dozen programming languages, including [[Go (programming language)|Go]], [[JavaScript]], [[Perl]], [[PHP]], [[Ruby (programming language)|Ruby]], [[Shell (programming language)|Shell]], [[Swift (programming language)|Swift]], and [[TypeScript]], though it is most effective in |
OpenAI claims that Codex is able to function in over a dozen programming languages, including [[Go (programming language)|Go]], [[JavaScript]], [[Perl]], [[PHP]], [[Ruby (programming language)|Ruby]], [[Shell (programming language)|Shell]], [[Swift (programming language)|Swift]], and [[TypeScript]], though it is most effective in Python.<ref name="OAI" /> According to ''[[VentureBeat]]'', demonstrations uploaded by OpenAI showed impressive [[coreference resolution]] capabilities and were able to create a [[browser game]] in JavaScript and generate data science charts using [[matplotlib]].<ref name="VB" /> |
||
OpenAI has demonstrated that Codex is able to interface with services and apps such as [[Mailchimp]], [[Microsoft Word]], [[Spotify]], and [[Google Calendar]].<ref name="VB" /><ref name="Verge">{{Cite news|last=Vincent|first=James|date=August 10, 2021|title=OpenAI can translate English into code with its new machine learning software Codex|work=[[The Verge]]|url=https://www.theverge.com/2021/8/10/22618128/openai-codex-natural-language-into-code-api-beta-access|access-date=2021-09-03}}</ref> [[Microsoft]] is reportedly interested in exploring Codex's capabilities.<ref name="Verge" /> |
OpenAI has demonstrated that Codex is able to interface with services and apps such as [[Mailchimp]], [[Microsoft Word]], [[Spotify]], and [[Google Calendar]].<ref name="VB" /><ref name="Verge">{{Cite news|last=Vincent|first=James|date=August 10, 2021|title=OpenAI can translate English into code with its new machine learning software Codex|work=[[The Verge]]|url=https://www.theverge.com/2021/8/10/22618128/openai-codex-natural-language-into-code-api-beta-access|access-date=2021-09-03}}</ref> [[Microsoft]] is reportedly interested in exploring Codex's capabilities.<ref name="Verge" /> |
||
Revision as of 03:38, 4 September 2021
OpenAI Codex is an artificial intelligence model developed by OpenAI. It parses natural language and converts it into code. It is used to power GitHub Copilot, a tool developed for Visual Studio Code.[1] Codex is a descendent of OpenAI's GPT-3 model, fine-tuned for use in programming applications.
OpenAI has released an API for Codex in closed beta.[1]
Capabilities
Based on GPT-3, a neural network trained on text, Codex has additionally been trained on 159 gigabytes of Python code from 54 million GitHub repositories.[2][3] OpenAI has stated that Codex can complete approximately 37% of requests and is meant to make human programming faster rather than replace it; according to OpenAI's blog, Codex excels most at "mapping [...] simple problems to existing code", which they describe as "probably the least fun part of programming".[4][5] According to a paper written by OpenAI researchers, when attempting each test case 100 times, 70.2% of prompts had working solutions.[6]
OpenAI claims that Codex is able to function in over a dozen programming languages, including Go, JavaScript, Perl, PHP, Ruby, Shell, Swift, and TypeScript, though it is most effective in Python.[1] According to VentureBeat, demonstrations uploaded by OpenAI showed impressive coreference resolution capabilities and were able to create a browser game in JavaScript and generate data science charts using matplotlib.[5]
OpenAI has demonstrated that Codex is able to interface with services and apps such as Mailchimp, Microsoft Word, Spotify, and Google Calendar.[5][7] Microsoft is reportedly interested in exploring Codex's capabilities.[7]
Issues
OpenAI demonstrations showcased flaws such as inefficient code and one-off quirks in code samples.[5] In an interview with The Verge, OpenAI chief technology officer Greg Brockman said that "sometimes [Codex] doesn't quite know exactly what you're asking" and that it can require some trial and error.[7] OpenAI researchers found that Codex struggles with multi-step and higher-level prompts, often failing or yielding counter-intuitive behavior. Additionally, they brought up several safety issues, such as over-reliance by novice programmers, biases based on the training data, and security impacts due to vulnerable code.[6]
VentureBeat has stated that because Codex is trained on public data, it could be vulnerable to data poisoning (intentional uploads of malicious code).[5] According to a study by researchers from New York University, approximately 40% of code generated by GitHub Copilot (which uses Codex) included glitches or other exploitable design flaws.[8] The Free Software Foundation has expressed concerns that code snippets generated by Copilot and Codex could unknowingly violate the terms of free software licenses, such as the GPL, which requires derivative works to be licensed under equivalent terms.[9]
References
- ^ a b c Zaremba, Wojciech (August 10, 2021). "OpenAI Codex". OpenAI. Retrieved 2021-09-03.
{{cite web}}: CS1 maint: url-status (link) - ^ Wiggers, Kyle (July 8, 2021). "OpenAI warns AI behind GitHub's Copilot may be susceptible to bias". VentureBeat. Retrieved 2021-09-03.
- ^ Alford, Anthony (August 31, 2021). "OpenAI Announces 12 Billion Parameter Code-Generation AI Codex". InfoQ. Retrieved 2021-09-03.
- ^ Dorrier, Jason (August 15, 2021). "OpenAI's Codex Translates Everyday Language Into Computer Code". SingularityHub. Retrieved 2021-09-03.
- ^ a b c d e Dickson, Ben (August 16, 2021). "What to expect from OpenAI's Codex API". VentureBeat. Retrieved 2021-09-03.
- ^ a b Chen, Mark; Tworek, Jerry; Jun, Heewoo; Yuan, Qiming; Pinto, Henrique Ponde de Oliveira; Kaplan, Jared; Edwards, Harri; Burda, Yuri; Joseph, Nicholas; Brockman, Greg; Ray, Alex (2021-07-14). "Evaluating Large Language Models Trained on Code". arXiv:2107.03374 [cs].
- ^ a b c Vincent, James (August 10, 2021). "OpenAI can translate English into code with its new machine learning software Codex". The Verge. Retrieved 2021-09-03.
- ^ Claburn, Thomas (August 25, 2021). "GitHub's Copilot may steer you into dangerous waters about 40% of the time – study". The Register. Retrieved 2021-09-03.
- ^ Krill, Paul (August 2, 2021). "GitHub Copilot is 'unacceptable and unjust,' says Free Software Foundation". InfoWorld. Retrieved 2021-09-03.