CNET

CNET’s AI Journalist Appears to Have Committed Extensive Plagiarism

Posted by

0


CNET’s AI-written articles aren’t just riddled with errors. They also appear to be substantially plagiarized.

Getty Images

The prominent tech news site CNET‘s attempt to pass off AI-written work keeps getting worse. First, the site was caught quietly publishing the machine learning-generated stories in the first place. Then the AI-generated content was found to be riddled with factual errors. Now, CNET‘s AI also appears to have been a serial plagiarist — of actual humans’ work.

The site initially addressed widespread backlash to the bot-written articles by assuring readers that a human editor was carefully fact-checking them all prior to publication.

Afterward, though, Futurism found that a substantial number of errors had been slipping into the AI’s published work. CNET, a titan of tech journalism that sold for $1.8 billion back in 2008, responded by issuing a formidable correction and slapping a warning on all the bot’s prior work, alerting readers that the posts’ content was under factual review. Days later, its parent company Red Ventures announced in a series of internal meetings that it was temporarily pausing the AI-generated articles at CNET and various other properties including Bankrate, at least until the storm of negative press died down.

Now, a fresh development may make efforts to spin the program back up even more controversial for the embattled newsroom. In addition to those factual errors, a new Futurism investigation found extensive evidence that the CNET AI’s work has demonstrated deep structural and phrasing similarities to articles previously published elsewhere, without giving credit. In other words, it looks like the bot directly plagiarized the work of Red Ventures competitors, as well as human writers at Bankrate and even CNET itself.

Jeff Schatten, a professor at Washington and Lee University who has been examining the rise of AI-enabled misconduct, reviewed numerous examples of the bot’s apparent cribbing that we provided. He found that they “clearly” rose to the level of plagiarism.

We asked Schatten what would happen if a student turned in an essay with a comparable number of similarities to existing documents with no attribution.

“They would be sent to the student-run ethics council and given the repeated nature of the behavior would almost certainly be expelled from the university,” he replied.

The bot’s misbehavior ranges from verbatim copying to moderate edits to significant rephrasings, all without properly crediting the original. In at least some of its articles, it appears that virtually every sentence maps directly onto something previously published elsewhere.

Take this excerpt, for instance, from a recent article by the CNET AI about overdraft protection:

How to avoid overdraft and NSF fees 

Overdraft fees and NSF fees don’t have to be a common consequence. There are a few steps you can take to avoid them.

And compare it to this verbiage from a previously published article in Forbes Advisor, a Red Ventures competitor:

How to Avoid Overdraft and NSF Fees 

Overdraft and NSF fees need not be the norm. There are several tools at your disposal to avoid them.

Sure, the bot’s version altered the capitalization and swapped out a few words for impressively lateral-minded synonyms — “the norm” becomes “a common consequence,” for instance, and “several tools” becomes “a few steps” — along with a few minor changes to the syntax. But apart from those semantic tweaks, the two sentences are nearly identical.

Here’s another excerpt from the same article by CNET‘s AI financial writer:

Sign up for low-balance alerts 

You may be able to receive low balance alerts from your bank’s mobile app, so you know if your account balance is dropping below a certain threshold.

Now compare it to this section from another previously published article, this one from The Balance, another Red Ventures competitor:

Sign Up for Low Balance Alerts 

You can sign up for low-balance alerts through most banks to alert you when your account hits a certain amount.

Again, it seems clear that the AI is simply parsing through and making small modifications to obscure the source.

Sometimes the similarities are almost comical in their lack of subtlety. Take the first sentence of this article, also published by CNET‘s AI:

Gift cards are an easy go-to when buying a present for someone.

And compare it to the first sentence of this previously published Forbes article:

Gift cards are an easy-to-please present for just about anyone.

The kicker on that one? Check out the almost imperceptible difference between those two articles’ headlines. Here’s the CNET AI’s title:

Can You Buy a Gift Card With a Credit Card?

And here’s what Forbes ran with for a headline:

Can You Buy Gift Cards With a Credit Card?

That’s right: the only difference is switching “Gift Cards” to a singular.

Here’s another example, from the same AI-generated CNET article about overdraft fees:

What is overdraft protection?

Overdraft protection is an optional feature offered by banks to prevent the rejection of a charge on a checking account with insufficient funds.

Which, it turns out, appears to be a word salad rephrasing of a line from this article on Investopedia, another Red Ventures competitor.

What Is Overdraft Protection?

Overdraft protection is an optional service that prevents the rejection of charges to a bank account… that are in excess of the available funds in the account.

The AI appears to sometimes also borrow language from writers at CNET‘s sister site Bankrate without giving credit. For example, look at this line from an article published by CNET‘s AI back in November: 

Becoming an authorized user can help you avoid applying for a card on your own, which is a major benefit if you currently have bad credit or no credit history.

And compare it to this wording, previously published by a Bankrate writer:

Becoming an authorized user also lets you avoid having to apply for a card on your own, which is a major benefit if you currently have bad credit or no credit history at all.

All told, a pattern quickly emerges. Essentially, CNET‘s AI seems to approach a topic by examining similar articles that have already been published and ripping sentences out of them. As it goes, it makes adjustments — sometimes minor, sometimes major — to the original sentence’s syntax, word choice, and structure. Sometimes it mashes two sentences together, or breaks one apart, or assembles chunks into new Frankensentences. Then it seems to repeat the process until it’s cooked up an entire article.

A current Red Ventures employee also reviewed examples of the bot’s seemingly lifted work.

“You ever copy your homework off of somebody,” they quipped, “but they told you to kind of rephrase it?”

“It poses the question of what kind of institutions do CNET and Bankrate want to be seen as,” they continued. “They’re just taking these articles and rephrasing a couple of things.”

Are you a current or former Red Ventures employee and want to share your thoughts about the company’s use of AI? Email us at tips@futurism.com. We can keep you anonymous.

In short, a close examination of the work produced by CNET‘s AI makes it seem less like a sophisticated text generator and more like an automated plagiarism machine, casually pumping out pilfered work that would get a human journalist fired.

Perhaps, at the end of the day, none of this should be terribly surprising. At their core, the way that machine learning systems work is that you feed in an immense pile of “training data,” process it with sophisticated algorithms, and end up with a model that can produce similar work on demand.

Investigators have sometimes found examples of AI plagiarizing its own training data. In 2021, for instance, researchers from Johns Hopkins University, New York University and Microsoft found that text-generating AIs “sometimes copy substantially, in some cases duplicating passages over 1,000 words long from the training set.”

As such, the question of exactly how CNET’s disastrous AI was trained may end up taking center stage as the drama continues to unfold. At a CNET company meeting late last week, The Verge reported at the time, the outlet’s executive vice president of content and audience refused to tell staff — many of them acclaimed tech journalists who have written extensively about the rise of machine learning — what data had been used to train the AI.

The legality of using data to train an AI without the consent of the people who created that data is currently being tested by several lawsuits against the makers of prominent image generators, and could become a flashpoint in the commercialization of the tech.

“If a student presented the equivalent of what CNET has produced for an assignment in my class, and if they did not cite their sources, then I would definitely count it as plagiarism,” said Antony Aumann, a philosophy professor at Northern Michigan University who recently made headlines when he discovered that one of his own students had submitted an essay generated using ChatGPT, after reviewing examples of the CNET AI’s similar phrasing to other outlets.

“Now, there is some dispute among academics about exactly what plagiarism is,” he continued. “Some scholars consider it a form of stealing; other scholars regard it as a kind of lying. I think of it in the latter way. Plagiarism involves representing something as your own that is in fact not your own. And that appears to be what CNET is doing.”

CNET did not respond to examples of the bot’s seemingly cribbed writing, nor to questions about this story.

In a sense, the relentless ineptitude of the company’s braindead AI probably obfuscates many of the thornier themes we’re likely to see emerge as the tech continues to spread into the workplace and information ecosystems.

Schatten, for instance, warned that issues around AI and intellectual property are likely to get more ambiguous and difficult to detect as AI systems continue to improve, or even as publishers start to experiment with more advanced systems that already exist (Red Ventures has declined to say what AI it’s using, though the editor-in-chief of CNET has said that it’s not ChatGPT.)

“The CNET example is noteworthy because whatever AI they were using was not drawing from the entirety of the internet and carefully coming up with a new mosaic, but rather just lifting more or less word for word from existing stories,” Schatten said. “But the more sophisticated AIs of today, and certainly the AIs of the future, will do a better job of hiding the origins of the material.”

“And especially once AIs are drawing from the writing of other AIs, which themselves are quoting AI (dark, I know) it might become quite difficult to detect,” he added.

In a practical sense, it seems increasingly obvious that CNET and Red Ventures deployed the AI system and started blasting its articles out to the site’s colossal audience without ever really scrutinizing its output. It wasn’t just that the architects of the program missed obvious factual errors, but that they appear never to have checked whether the system’s work might have been poached.

And to be fair, why would they? As The Verge reported in a fascinating deep dive last week, the company’s primary strategy is to post massive quantities of content, carefully engineered to rank highly in Google, and loaded with lucrative affiliate links.

For Red Ventures, The Verge found, those priorities have transformed the once-venerable CNET into an “AI-powered SEO money machine.”

That might work well for Red Ventures’ bottom line, but the specter of that model oozing outward into the rest of the publishing industry should probably alarm anybody concerned with quality journalism or — especially if you’re a CNET reader these days — trustworthy information.

CNET’s Article-Writing AI Is Already Publishing Very Dumb Errors

Posted by

0


CNET is now letting an AI write articles for its site. The problem? It’s kind of a moron.

Getty Images

Last week, we reported that the prominent technology news site CNET had been quietly publishing articles generated by an unspecified “AI engine.”

The news sparked outrage. Critics pointed out that the experiment felt like an attempt to eliminate work for entry-level writers, and that the accuracy of current-generation AI text generators is notoriously poor. The fact that CNET never publicly announced the program, and that the disclosure that the posts were bot-written was hidden away behind a human-sounding byline — “CNET Money Staff” — made it feel as though the outlet was trying to camouflage the provocative initiative from scrutiny.

After the outcry, CNET editor-in-chief Connie Guglielmo acknowledged the AI-written articles in a post that celebrated CNET‘s reputation for “being transparent.”

Without acknowledging the criticism, Guglielmo wrote that the publication was changing the byline on its AI-generated articles from “CNET Money Staff” to simply “CNET Money,” as well as making the disclosure more prominent.

Furthermore, she promised, every story published under the program had been “reviewed, fact-checked and edited by an editor with topical expertise before we hit publish.”

That may well be the case. But we couldn’t help but notice that one of the very same AI-generated articles that Guglielmo highlighted in her post makes a series of boneheaded errors that drag the concept of replacing human writers with AI down to earth.

Take this section in the article, which is a basic explainer about compound interest (emphasis ours):

“To calculate compound interest, use the following formula:

Initial balance (1+ interest rate / number of compounding periods) ^ number of compoundings per period x number of periods 

For example, if you deposit $10,000 into a savings account that earns 3% interest compounding annually, you’ll earn $10,300 at the end of the first year.“

It sounds authoritative, but it’s wrong. In reality, of course, the person the AI is describing would earn only $300 over the first year. It’s true that the total value of their principal plus their interest would total $10,300, but that’s very different from earnings — the principal is money that the investor had already accumulated prior to putting it in an interest-bearing account.

“It is simply not correct, or common practice, to say that you have ‘earned’ both the principal sum and the interest,” Michael Dowling, an associate dean and professor of finance at Dublin College University Business School, told us of the AI-generated article.

It’s a dumb error, and one that many financially literate people would have the common sense not to take at face value. But then again, the article is written at a level so basic that it would only really be of interest to those with extremely low information about personal finance in the first place, so it seems to run the risk of providing wildly unrealistic expectations — claiming you could earn $10,300 in a year on a $10,000 investment — to the exact readers who don’t know enough to be skeptical.

Another error in the article involves the AI’s description of how loans work. Here’s what it wrote (again, emphasis ours):

“With mortgages, car loans and personal loans, interest is usually calculated in simple terms.

For example, if you take out a car loan for $25,000, and your interest rate is 4%, you’ll pay a flat $1,000 in interest per year.”

Again, the AI is writing with the panache of a knowledgeable financial advisor. But as a human expert would know, it’s making another ignorant mistake.

What it’s bungling this time is that the way mortgages and auto loans are typically structured, the borrower doesn’t pay a flat amount of interest per year, or even per monthly payment. Instead, on each successive payment they owe interest only on the remaining balance. That means that toward the beginning of the loan, the borrower pays more interest and less principal, which gradually reverses as the payments continue.

It’s easy to illustrate the error by entering the details from the CNET AI’s hypothetical scenario — a $25,000 loan with an interest rate of 4 percent — into an auto loan amortization calculator. The result? Contrary to what the AI claimed, there’s never a year when the borrower will pay a full $1,000, since they start chipping away at the balance on their first payment.

CNET‘s AI is “absolutely” wrong in how it described loan payments, Dowling said.

“That’s just simply not the case that it would be $1,000 per year in interest,” he said, “as the loan balance is being reduced every year and you only pay interest on the outstanding balance.”

The problem with this description isn’t just that it’s wrong. It’s that the AI is eliding an important reality about many loans: that if you pay them down faster, you end up paying less interest in the future. In other words, it’s feeding terrible financial advice directly to people trying to improve their grasp of it.

The AI made yet another gaffe when it attempted to describe certificates of deposit, better known as CDs, which are financial products that offer interest, but typically discourage withdrawing the funds before a set period has elapsed (once more, emphasis ours):

“Note that a high-yield savings account or money market account may offer interest that compounds daily, weekly or monthly. But a one-year certificate of deposit only compounds once, after the initial deposit reaches maturity.”

This one is just straight-up false. For instance, here’s a one-year CD by Chase Bank that compounds daily. And here’s one by Capital One that compounds monthly.

All three screwups, each of which the AI presented with the easy authority of an actual subject matter expert, highlight a core issue with current-generation AI text generators: while they’re legitimately impressive at spitting out glib, true-sounding prose, they have a notoriously difficult time distinguishing fact from fiction.

For an editor, that’s bound to pose an issue. It’s one thing to work with a writer who does their best to produce accurate work, but another entirely if they pepper their drafts with casual mistakes and embellishments. BuzzFeed News perfectly illustrated that risk this week, when a reporter there used ChatGPT to generate a story about CNET‘s secretive use of AI — only to find that she “had to rewrite the prompt a few times to get it to stop inserting factual errors.”

Another issue that may be at play here is well known in the separate AI-inflected field of self-driving cars. Researchers have found that human safety drivers, tasked with sitting behind the wheel of an autonomous vehicle to take over if it malfunctions, tend to quickly lose focus when they don’t have to actively work the controls. The same dynamic may be at play when an editor is put in charge of approving a deluge of AI-generated explainers: in the face of endless synthetic writing, maybe it makes sense that human editors start to go on autopilot themselves.

Everyone makes mistakes, so we’re certainly sympathetic. But in these early days of CNET‘s AI experiment — nevermind in a piece published the same day that the site’s editor went public in response to a storm of criticism — you’d expect the editors tasked with monitoring the AI to be on their highest alert.

If these are the sorts of blunders that slip through during that period of peak scrutiny, what should we expect when there aren’t so many eyes on the AI’s work? And what about when copycats see that CNET is getting away with the practice and start filling the web with their own AI-generated content, with even fewer scruples?

It’s also worth asking what readers actually want: financial advice from a real human with real financial concerns, or logorrhea from a bot that’s been trained to rehash existing financial writing with no financial stake of its own.

Dowling said that while he’s optimistic about the potential of AI in general, he suspects that an algorithm like CNET‘s lack of personal perspective or “insights that go beyond mere summary” will keep it from producing genuinely interesting work.

“People already approach finance reading with an advance sense of boredom and reluctance — will ChatGPT just embed those negative features even deeper in finance writing?” he asked.

After Futurism reached out to CNET about the errors, staff there issued a lengthy correction to the article and edited the text to address all three mistakes.

Staff at CNET also seemingly identified a fourth error by the AI, which they also described in the correction, regarding the distinction between Annual Percentage Rate (APR) and Annual Percentage Yield (APY).

A CNET spokesperson provided Futurism with a brief statement about the corrections.

“We are actively reviewing all our AI-assisted pieces to make sure no further inaccuracies made it through the editing process, as humans make mistakes, too,” they said. “We will continue to issue any necessary corrections according to CNET’s correction policy.”

The spokesperson didn’t respond to a question about CNET‘s confidence in the other articles the AI has published to its site. After we reached out, however, a new message appeared at the top of almost every piece the AI has published, dating back to November.

“Editors’ note: We are currently reviewing this story for accuracy,” reads the message. “If we find errors, we will update and issue corrections.”

It’s worth pointing out, as Platformer‘s Casey Newton did this week, that CNET‘s AI-generated finance articles arguably only exist in the first place because they’re trying to manipulate Google’s algorithm for profit. Countless better explanations of compound interest already exist; CNET‘s strategy is simply to publish large volumes of cheaply produced text, carefully optimized to float to the top of search results, in a bid to capture the monetizable eyeballs of the financially curious.

“Over time, we should expect more consumer websites to feature this kind of ‘gray’ material: good-enough AI writing, lightly reviewed (but not always) by human editors, will take over as much of digital publishing as readers will tolerate,” Newton wrote. “The quiet spread of AI kudzu vines across CNET is a grim development for journalism, as more of the work once reserved for entry-level writers building their resumes is swiftly automated away.”

In other words, it’s not just AI that’s the issue here. It’s that AI is maturing at a moment when the journalism industry has already been hollowed out by a decades-long race to the bottom — a perfect storm for media bosses eager to cut funding for human writers.

Five things Microsoft must do for Windows 8 in 2013.

Posted by

0


MS_Windows_8_Surface_event_NY_Panay_1_610x378Windows_8_RTM_2_Charms_610x343

Reactions to Windows 8‘s launch mirrored the operating system’s own split personality. Here’s what Microsoft needs to do to keep the OS from getting left on the showroom floor.

Microsoft’s ambitious Windows 8 gamble may have launched this past October, but it’s 2013 that will make or break the new operating system. I have five recommendations that Microsoft should implement sooner rather than later to keep Windows 8 from going the way of Vista.

Make the case for Windows RT
“That’s right, it filets, it chops, it dices, slices, never stops, lasts a lifetime, mows your lawn, and it mows your lawn and it picks up the kids from school…” —Tom Waits, “Step Right Up”

Waits wasn’t talking about Windows RT when he wrote and recorded “Step Right Up” in the mid-’80s, but he could’ve been. Microsoft wants the tablets that run the OS to be unifying devices that are portable like a tablet but powerful enough for the heavy lifting of Microsoft Office. Claiming that the OS can step up to that challenge, and actually proving that it can, are not the same thing.

Here’s the problem with Windows RT: Even after writing CNET’s FAQ on Windows RT, I still have problems clearly explaining what it is and why people should want it. It’s “Windows 8 Lite,” but it’s so much more complex than that. Sure, the Surface is a nice piece of hardware, but besides its utility as a tablet-and-skateboard combo it’s a hard sell.

I have a semibaked theory that Windows RT will become Windows 9, especially because of its ability to run on lower-powered, more secure ARM chips, but right now RT is closer to being the next Kin than the next Xbox.

Focus on apps
Though some Microsoft defenders point out that it took Google years to bulk up Android‘s app catalog to 500,000-plus titles, Redmond doesn’t have that kind of time when it comes to Windows 8.

Windows is not some in-development mobile operating system; it’s the mature senior statesman of the computing world. It’s on more computers than any other OS, and that’s not going to change anytime soon. We know Microsoft wants the world to move as quickly as possible to Windows 8 — there’s no other explanation for the soon-to-expire $39.99 upgrade and the push for new, interesting, touch-screen hardware.

While it’s true that Windows 8 can run legacy software just fine in Desktop mode, Metro apps are what will sell people. Some good apps currently available demonstrate the possibilities of Metro, but they don’t offer a compelling reason to change your entire work flow.

The strength of iOS is that Apple’s operating system is the cleanest around. Android basks in the glow of Google’s best apps and services, from Gmail to Goggles to Translate to search. The Windows 8 app experience has yet to be defined, which could benefit Microsoft in that it has an open canvas to paint on.

The bad? Don’t expect competitors to look the other way as Microsoft refines its app pitch to developers.

Convertibles and hybrids need a Surface, too
The Surface hardware went a long way toward drumming up interest in Windows 8 and Windows RT, not to mention a lot of sturm und drang from Microsoft’s hardware partners. While it could be interesting to see a Redmond-designed convertible or hybrid laptop, it’s not strictly necessary. But what the burgeoning, occasionally confusing category does require is a hybrid or convertible that Microsoft can point to and exclaim to the public, “This!”

It may not want to, but right now all that we’ve seen are oversize tablet-tops with hinges. You can’t easily tell people why they must have a new category of hardware without a signature device.

Wherefore art thou, settings and preferences?
Settings aren’t sexy, but they shouldn’t be confusing, either. Microsoft ought to make some decisions, and fast, about cleaning up the confusing mess of its under-the-hood options.

Sometimes they’re behind the Settings charm in Metro. Other times they’re buried in some Desktop mode window. Currently, I find it easiest to simply start typing for what I’m looking for, and let the powerful search tool do the hard work. But if Microsoft wants Windows 8 to have long-standing appeal for nonexperts, it’s going to have to demystify this stuff.

Get people and businesses excited about Windows 8
Microsoft has done itself a great disservice by coming up with a fairly interesting, unique approach to the ecosystem problem, and then letting substandard marketing heighten people’s questions and uncertainties.

Solving the above problems alone won’t work without helping people realize what’s so great about Windows 8. And without the massive license buys that businesses can provide, Windows 8 will struggle in a consumer marketplace that is increasingly turning to Macs to solve its problems.

How Microsoft can best do that I’ll leave to greater marketing minds than myself. On some level, though, it would seem easiest to have compelling hardware that people want to use. The interest in the Surface is a step in the right direction, just as Samsung’s Galaxy S3 and Google’s Nexus 7 did wonders for Android. Maybe there’s a killer “laptablet” coming at the beginning of next year, but there’s little doubt that Windows 8 has a hard path to trek in 2013.

By Seth Rosenblatt

Seth Rosenblatt is a senior editor at CNET, and has written about nearly every category of software and app available. At CNET since 2006, he currently focuses on browsers, security, and operating systems, with occasional forays into tech and pop culture.

Source:CNET