Man vs. Machine 2.0

All last month, Gauss Algorithmic and Alex Alderman teamed up to test the Gauss Summarizer — an AI-powered tool that reduces any body of text into an adjustable-lengthed summary. The algorithm and the professional copywriter went head-to-head over four rounds, seeing who could summarize articles better. Read more about the Man 👨‍🏫 vs. Machine 👨‍💻 demo below!

A quick recap...

The demo was broken up into four rounds. For each round, both contestants summarized the same article, and the public voted on which article was more useful to them. The first two sounds focused on a short summary, and the second pair required both participants to produce something a bit longer. You can check out the four rounds for yourself: Round 1, Round 2, Round 3, Round 4.

Results

As we noted in our halfway report, the first half of the contest had mixed results. Both man and machine received votes, and we talked about some advantages for each side (more on that below). The second half of the contest was a little less close 😤 — with the human seeming to have the preferred summaries for the third and fourth articles. This provides a helpful benchmark about what the strengths are of the algorithm, and how it can be used most efficiently. Let’s take a closer look at rounds three and four:

Round 3 analysis

Alex Alderman

It’s time for all of us to start wearing better masks. Simple, home-made cloth masks were a good temporary solution at the beginning of the pandemic, but now that it’s almost a year later, there should be enough medical-grade masks for everyone to wear. Even though a cloth mask is much better than nothing, it’s time for all of us to wear the best masks possible, especially now that the new, more deadly, B-variant of COVID-19 is spreading across the U.S. and Europe.

Gauss Summarizer
It's past time for better solutions to be available to the public, authors say. Cloth masks, especially homemade ones, were supposed to be a stopgap measure. America is swamped with fraudulent medical-grade masks, some of which are only 1 percent effective. A new variant of the coronavirus, known as the B variant, is threatening Europe and the U.S. The authors ask: Why can’t we? Fixing this problem is more urgent now that a new variant of the coronavirus, known as the B, is threatening Europe and the U.S.

These two summaries aren’t too different. There are a couple less-natural sounding sentence structures from the algorithm that kind of gave away who wrote what, but overall the key information is present in both. In fact, we think that maybe the slightly clumsy sentences may hide the fact that the algorithm included an important piece of information about fraudulent masks — something that Alex left out. We asked him why he didn’t include it:

“You know, I just didn’t have the personal experience of having to deal with fake masks. Like most people in Prague, I started off by using a homemade cloth mask instead of trying to find a medical grade mask. So when I was reading about the need for new masks, I completely focused on what was relevant to me and the people that I know: that the cloth masks that we have been using are less effective, and that we should upgrade to a medical-grade version because they’re more effective. I guess the part about fake masks just didn’t register as super important to me. You’re not going to put this in the blog, are you?”

No, Alex. We’re not going to put this in the blog. 😜 Don’t worry, he decidedly wins the next round 😓 — let’s have a look:

Round 4 analysis

Alex Alderman
Canadian astronomer Robert Weryk believes he saw an alien while he was reviewing images captured by the Pan-starrs1 telescope, located in Hawaii. He saw a bright object, which was traveling in such a way that couldn’t be explained away by traditional explanations of unidentified space objects. He named it “Oumuamua.” When he reported his findings with his scientific peers, they were quite frustrated that they had to explain to the public that just because we don’t know exactly what it is doesn’t mean that it’s an alien.

Gauss Summarizer
A Canadian astronomer named Robert Weryk was reviewing images captured by a telescope known as Pan-starrs1 when he noticed something strange. The bright dot, astronomers concluded, was something never before seen . Oumuamua’s weird motion couldn’t be accounted for by a collision with another object, or by interactions with the solar wind, or by a phenomenon that’s known, after a nineteenth-century Polish engineer, as the Yarkovsky effect.

Above we can see a clear example of where the human touch came in very handy. Alex knew that “Oumuamua” needed to be explained. The algorithm just knew it was important — but didn’t have the insight that the average reader wouldn’t understand what it was referring to. You can also see that when the summaries get longer, (talented) humans can piece together multiple pieces of information in a cohesive sentence that flow together nicely. The algorithm just kind of threw all the silverware into the kitchen drawer without worrying about whether the spoons and the forks went nicely into their separate places. The algorithm is clearly not married. 🤖🙅‍♀️

Human advantages

So, throughout the four rounds we found some things that humans do better than machines: mostly they understand what the average reader understands, they provide better context, and they (for now) are better at making nicely flowing paragraphs. Not bad! But, before we delete our Summarizer forever … let’s talk about what advantages it has.

Robot advantages

Again, we can’t stress enough how much faster our Summarizer is. It produced its summaries over 500x faster than the professional — not bad, right! Also, even though the “human touch” can be advantageous in ways we mentioned above, it can also be a detriment. Not only can subjective experience cause a writer to leave out important information (like the issue with fake masks in round three), but as we mentioned in the previous write-up, human bias can be a huge problem — especially with news that evokes strong emotions like politics. Obviously, our algorithm doesn’t have that problem. 💪

How can Gauss help you?

So, what problems can the Gauss Summarizer solve for you? We pride ourselves in custom-designing AI-powered solutions for our clients, and we’d be happy to turn our Summarizer into the missing tool in your company’s toolkit. Our team can not only adjust it to fit your exact use-case, but we can work together to brainstorm how to harness the power of AI to turbocharge your business — whether that involves this product, or something else. Reach out to us, and we’d be happy to advise you!

Need an English copywriter?

Last thing, if you enjoyed this demo, you may want to consider hiring Alex to create engaging content for your company as well! He runs his business through LinkedIn, and you can message him directly there — here’s his profile. He specializes in web content (like this!), social media content, web copy, UX writing, video scripts, etc., etc. He lives in Prague, can read Czech, and has years of experience as a copywriter. He also excels in working 1-on-1 with executives to improve their business English, presentation skills, and negotiation skills.