All posts by zapier

New AI classifier for indicating AI-written text

New AI classifier for indicating AI-written text

We’re launching a classifier trained to distinguish between AI-written and human-written text.

We’ve trained a classifier to distinguish between text written by a human and text written by AIs from a variety of providers. While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations for false claims that AI-generated text was written by a human: for example, running automated misinformation campaigns, using AI tools for academic dishonesty, and positioning an AI chatbot as a human.

Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems.

We’re making this classifier publicly available to get feedback on whether imperfect tools like this one are useful. Our work on the detection of AI-generated text will continue, and we hope to share improved methods in the future.

Try our work-in-progress classifier yourself:

Try the classifier

Limitations

Our classifier has a number of important limitations. It should not be used as a primary decision-making tool, but instead as a complement to other methods of determining the source of a piece of text.

  1. The classifier is very unreliable on short texts (below 1,000 characters). Even longer texts are sometimes incorrectly labeled by the classifier.
  2. Sometimes human-written text will be incorrectly but confidently labeled as AI-written by our classifier.
  3. We recommend using the classifier only for English text. It performs significantly worse in other languages and it is unreliable on code.
  4. Text that is very predictable cannot be reliably identified. For example, it is impossible to predict whether a list of the first 1,000 prime numbers was written by AI or humans, because the correct answer is always the same.
  5. AI-written text can be edited to evade the classifier. Classifiers like ours can be updated and retrained based on successful attacks, but it is unclear whether detection has an advantage in the long-term.
  6. Classifiers based on neural networks are known to be poorly calibrated outside of their training data. For inputs that are very different from text in our training set, the classifier is sometimes extremely confident in a wrong prediction.

Training the classifier

Our classifier is a language model fine-tuned on a dataset of pairs of human-written text and AI-written text on the same topic. We collected this dataset from a variety of sources that we believe to be written by humans, such as the pretraining data and human demonstrations on prompts submitted to InstructGPT. We divided each text into a prompt and a response. On these prompts we generated responses from a variety of different language models trained by us and other organizations. For our web app, we adjust the confidence threshold to keep the false positive rate very low; in other words, we only mark text as likely AI-written if the classifier is very confident.

Impact on educators and call for input

We recognize that identifying AI-written text has been an important point of discussion among educators, and equally important is recognizing the limits and impacts of AI generated text classifiers in the classroom. We have developed a preliminary resource on the use of ChatGPT for educators, which outlines some of the uses and associated limitations and considerations. While this resource is focused on educators, we expect our classifier and associated classifier tools to have an impact on journalists, mis/dis-information researchers, and other groups.

We are engaging with educators in the US to learn what they are seeing in their classrooms and to discuss ChatGPT’s capabilities and limitations, and we will continue to broaden our outreach as we learn. These are important conversations to have as part of our mission is to deploy large language models safely, in direct contact with affected communities.

If you’re directly impacted by these issues (including but not limited to teachers, administrators, parents, students, and education service providers), please provide us with feedback using this form. Direct feedback on the preliminary resource is helpful, and we also welcome any resources that educators are developing or have found helpful (e.g., course guidelines, honor code and policy updates, interactive tools, AI literacy programs).

Contributors

Michael Lampe, Joanne Jang, Pamela Mishkin, Andrew Mayne, Henrique Ponde de Oliveira Pinto, Valerie Balcom, Michelle Pokrass, Jeff Belgum, Madelaine Boyd, Heather Schmidt, Sherwin Wu, Logan Kilpatrick, Thomas Degry

source https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/

The Responsible AI Manifesto for Marketing and Business

The Responsible AI Manifesto for Marketing and Business

More Intelligent. More Human.TM

In the process of making software more intelligent, AI has the potential to make brands more human by enabling us to focus increased time and energy on communications, creativity, culture, community, and the human condition.

AI should make us better people, professionals, and organizations. However, this will not happen without a continuous focus on the responsible application of AI across all business functions.

from Marketing AI Institute | Blog https://ift.tt/Keckh9H
via IFTTT

OpenAI and Microsoft Extend Partnership

We're happy to announce that OpenAI and Microsoft are extending our partnership.

This multi-year, multi-billion dollar investment from Microsoft follows their previous investments in 2019 and 2021, and will allow us to continue our independent research and develop AI that is increasingly safe, useful, and powerful.

In pursuit of our mission to ensure advanced AI benefits all of humanity, OpenAI remains a capped-profit company and is governed by the OpenAI non-profit. This structure allows us to raise the capital we need to fulfill our mission without sacrificing our core beliefs about broadly sharing benefits and the need to prioritize safety.

Microsoft shares this vision and our values, and our partnership is instrumental to our progress.

  • We've worked together to build multiple supercomputing systems powered by Azure, which we use to train all of our models. Azure's unique architecture design has been crucial in delivering best-in-class performance and scale for our AI training and inference workloads. Microsoft will increase their investment in these systems to accelerate our independent research and Azure will remain the exclusive cloud provider for all OpenAI workloads across our research, API and products.
  • Learning from real-world use – and incorporating those lessons – is a critical part of developing powerful AI systems that are safe and useful. Scaling that use also ensures AI’s benefits can be distributed broadly. So, we've partnered with Microsoft to deploy our technology through our API and the Azure OpenAI Service — enabling enterprise and developers to build on top of GPT, DALL·E, and Codex. We’ve also worked together to build OpenAI’s technology into apps like GitHub Copilot and Microsoft Designer.
  • In an effort to build and deploy safe AI systems, our teams regularly collaborate to review and synthesize shared lessons – and use them to inform iterative updates to our systems, future research, and best practices for use of these powerful AI systems across the industry.

We look forward to continued collaboration and advancing this progress with Microsoft.

source https://openai.com/blog/openai-and-microsoft-extend-partnership/