Machine learning and AI is a rapidly evolving space. In years past, it was the domain of big companies investing R&D dollars into AI, searching for breakthroughs to determine if and how AI could be used in the future. Now, we are seeing machine learning and artificial intelligence making its way into daily use marketing tools that claim everything from reducing the time spent on tasks all the way up to complete automation.
Machine learning has worked its way into various paid search tools, for example, that use algorithmic learning to try to keep your PPC campaign bid prices as low as possible, while keeping the amount of traffic you receive high.
One particular area AI is gaining traction is in generating content. This is not new, and some of these robot generated content tools have been around since the the 2010s. However, they produced very low quality content, often including typos.
The Rise of AI in Generating Content
Recently there have been a number of companies on the market that claim they can now create high quality, readable content. They often refer to this as “human quality content.”
Some of these technologies have been around for a while, such as the chatbot program which is used in marketing automation. It’s an AI based tool programmed to answer questions and provide helpful information about your products or services on websites, or used in support.
Are these tools ready to be added to your marketing workflow in a daily capacity? Can they product they type of what they claim? We decided to test them.
The Use of Marketing Tools in our Agency
Working in a growing digital marketing agency means that you have to be ready to pivot to another project quickly, sometimes multiple times a day. Most people that work in an agency will be actively working on 5 – 10 projects at the same time, and may be asked to weigh in on others from time-to-time, as well.
As a results, time is the most valuable commodity we manage in our agency. We want to provide the best thinking and highest level of work to any client. Not only that, we want them to have the perception they are our only client. We never want them to feel like they are being juggled in any way between other work.
People with experience and a good brain will always be the most valuable resource we have. But if we can augment productivity with smart, effective tools, we are happy to make the investment. The key to any tool we invest in is that it has to be able leverage a person’s expertise, and provide real, tangible value to a client.
One example would be SEMRush. We use it for a variety of things, but one key use is its ability to index a client’s site to measure it for SEO. Our SEO experts could do the same thing manually, but SEMRush can do it faster, and it allows our experts to focus on analysis, prioritization and planning. In other words, it does not replace our SEO subject matter experts–it allows them to apply their skill in a way that is more meaningful and helpful to our clients.
Another area that we spend a lot of time on is creating high quality content. We do this in our own articles section of our site, where we share what we have learned and how we go about solving marketing problems, but also creating a lot of content for clients. So it made sense that we evaluate some of these copywriting tools to see if they can do what they claim.
What is the Difference Between Machine Learning and Artificial Intelligence (AI)?
Machine learning and AI have become interchangeable terms, however they are not the same. Machine learning is the process of teaching a computer to learn from patterns in data and use them to make predictions. AI is the possible outcome of machine learning–a computer capable of cognitive functions, including visual perception, speech recognition, decision making, translation between languages, and, in this case we are discussing, generating human quality content.
In most cases, when people reference AI, they are really taking about machine learning. This is directing programs to pattern analysis and rule evaluations to learn from data without being explicitly programed to do so. AI is more akin to leaps in judgement and decision making, more along the lines of human style thinking. Autonomous programs that can learn about subjects without almost no direction at all.
The Role of a Copywriter on a Project
I want to be clear that we are not attempting to reduce or replace a copywriter in the role of creating great content. Our evaluation was to see if some of these copy creation tools could assist our copywriters, similar to SEMRush‘s ability to assist our SEO experts.
Copywriters are marketing professionals who create content for a project. They also often act as marketing strategists that are trying to determine the best way to approach a target audience. They want to uncover what types of marketing messages will appeal and get people to take action.
The role of a good copywriter is much larger than just assembling words. They have to understand how your audience thinks, the problems they are trying to solve, how to get them to flow from one page to the next on your site, and how to get someone to take action. That is a skill well beyond just the construction of sentences.
Are these AI copywriting tools ready for this? Well…. keep reading. We testing them and we have some clear thoughts on this matter.
AI Helped Write this Article
We have tested a variety of copywriting tools in a variety of situations. We tested them in the writing of Google ads, Facebook ads, title tags, meta descriptions, blog ideas, writing product descriptions, and others.
We also used them to help write this very article. We put them to the test and decided to use them to help organize and write out our thoughts on the matter.
The AI Copy Tools We Tested
There are a growing number of tools on the market now that claim to use sophisticated AI to analyze and generate content. There may be some we are not aware of, but we tested some of the most popular.
- Jasper.ai
- Copysmith.ai
- copy.ai
These tools all have their own take on how they generate content, but generically they claim to be marketing automation services that uses machine learning to generate copy for ad campaigns, e-newsletters, blog posts, and a wide variety of other types of content. The companies say their tools can write conversion optimized copy in under 15 seconds using AI algorithms which improve with each new piece of content.
It is important to note that these are three tools in a growing market with many more options available.
How We Tested The AI Copywriting Tools
We used the three tools on a variety of content creation tasks. While we will get into the specifics of each tool, they generally operate in the same way. You select the type of content you are working on (Google ads, product descriptions, blog intros, meta descriptions, etc), provide the tool some information, such as a few keywords and a description, and they will provide anywhere from 3 – 10 results for you to choose from and use.
In our test, we tried them out with creating Google Ad Headlines, Google Ad descriptions, and product descriptions. We tried them out on a variety of other content types over the past month, but those are the three content types I will focus on in this article.
Side-by-Side Comparison
Here is are some comparisons of content each tool produced. I picked a few standard content types that all three claim they can easily create content. With that in mind, I used each tool to generate a Google Ad for a sample product, and a product description.
For this, I set each tool up with the same information. I used a fairly typical product one would find on an ecommerce site: a men’s crewneck t-shirt. I asked each tool to generate the content types I mentioned above, indicating a friendly tone of voice. Here is what I got.
Google Ads Headlines
For each of the three tools, you have to provide information for them to base the ad headlines on. One key thing to remember is that Google Ads has a character limit for headlines– 30 characters or less. None of the tools tested had an obvious indicator that they were trying to stay inside or work around that character limit.
Copy.ai
Google Ad Headlines
- casual shirt for men
- quality tee shirt
- casual everyday clothes.
- Shop Popular & Trending Styles
Copysmit.ai
Google Ad Headlines
- Cool T-shirt
- Mens Tee Shirt
- Comfortable T-shirt
- Funny T-Shirts
Jasper.ai
Google Ad Headlines
- New collection for men
- Wear Your Passion
- Guaranteed To Be Comfortable
The Verdict: Mixed
After working with these tools for awhile, they seems to have a harder time with short content. I noticed this when testing them on crafting product headlines, too.
- Copy.ai gave me headline options there were too long, exceeding Google’s character limits set in Google Ads.
- Copysmith.ai provided headlines that were rather simple, and didn’t vary much. Even if they weren’t perfect, it would be hard to use them as inspiration to write my own headlines.
- Jasper.ai provided the most variation in headlines. Even if they weren’t perfect, I could use them as inspiration to help me come up with more options on my own.
Google Ad Descriptions
Similar to ad headlines, Google Ads has a character limit for all ad descriptions of 90-characters. None of the tools testing seemed to be attempting to work with the 90-character limit. Some descriptions generated were pretty short, others longer than 90-characters.
Copy.ai Google Ad Descriptions
- Men’s t-shirt with spread collar. Super soft and made to last.
- It is perfect for wearing on summer days.
- A great t-shirts for a great time at a great price.
- A wardrobe staple. The classic striped crewneck tee shirt
Copysmit.ai Google Ad Descriptions
- Sip coffee on your porch, catch a tan in the sun, or walk along some beach while wearing our shirt.
- Summer is for relaxing and taking it easy. We invite you to be comfortable in our shirt.
- Hi there, nice to meet you.
- Reasonably priced. Designed in Europe.
Jasper.ai Google Ad Descriptions
- Men’s t-shirt is perfect for summer. Lightweight and comfortable for going out or staying in.
- You’ve heard about dress shirt layering. Now it’s time to try it out for yourself with this 3-pack of our classic men’s crewneck tee.
- Our men’s crewneck tee is on sale. Wear it for dressing up or a casual night on the town.
The Verdict: Two Bad, One Good
If you are a seasoned (or certified) Google Ads marketer, these tools may offer little help here. Another thing to consider is that Google Ads has a copy policy regarding the types of things you can say in ads and things you can’t. While none of the tools seemed to generate Google Ad descriptions that would generate policy violations, it doesn’t appear they are taking ad policy into consideration. It would be up to you, the user, to know and understand the ad copy policies to course correct any generated descriptions.
- Copy.ai Google ad descriptions too short to really work with. In order to make them work, I would have to combine two or even three of their suggestions to create one workable ad description.
- Copysmith.ai provided a single decent ad description. Two were too short, and one was completely a nonsensical “nice to meet you” that made me wonder if the tool was even working.
- Jasper.ai provided the most interesting headlines, however they exceeded the 90-character limit imposed by Google for ad descriptions. However, a few quick edits and I could adjust them to work.
Product Descriptions
For this sample set I used each tool to create a product description for our typical men’s t-shirt. I provided the same set of information to each tool: men’s crewnew t-shirt, seeking a friendly tone of voice.
Copy.ai Product Descriptions
- NextWorks is a men’s crewneck tee shirt, made with 100% combed cotton. It has a friendly tone of voice.
- Soft and comfortable for summer, the NextWorks crewneck tee shirt is a light and friendly tone of voice with vintage style Bébas embroidery, modeled on an original silk shirt from the 1920s. Poly/cotton.
- This NextWorks t-shirt is perfect for wearing on a summer day, or running around town. The friendly tone of voice is sure to amuse and delight anyone who hears it. This super soft t-shirt fits true to size, and is also available in several other colors.
- The NextWorks men’s crewneck tee shirt is the perfect solution for a variety of uses. It works equally well as a fashion piece or as an adornment to a costume. What better way to update your wardrobe with products from NextWorks?
- The NextWorks t-shirt for men is perfect for the summer. Lightweight and soft, this short-sleeve shirt has a fun way of talking to you while you’re wearing it. It also features tag-free comfort and a versatile design that can be worn tucked in or out. The fabric is pre shrunk with no ironing needed and no fading so there’s less work for you. Featuring a stylish cut, this tee is the perfect year round wardrobe staple.
Copysmith.ai Product Descriptions
- NextWorks Tee is a comfortable, soft and lightweight men’s crewneck tee shirt made with super-soft single jersey fabric. It’s a perfect addition to your casual wardrobe.
- The NextWorks T-shirt is the ultimate in casual comfort. This classic tee will become your favorite go-to as soon as you put it on. It’s lightweight, soft, and made from 100% combed cotton for your utmost comfort.
- If you’re looking for an item that’s at the forefront of its field, then this is it. It’s sure to keep you at the cutting edge of fashion and ahead of the curve in terms of style. It is truly a unique piece and one of a kind.
- Our tee shirt is comfortable, soft and lightweight. It’s perfect for daily wear in or out of the office.
- The NextWorks Tee is the perfect grab-and-go shirt for any occasion. Featuring a classic crew neckline and slim fit. Made from a soft, lightweight cotton blend jersey fabric, it has a comfortable, relaxed feel that’s ideal for work or play. Dress it up or down with any accessory you choose—the possibilities are endless!
- NextWorks Tee is a soft, lightweight, and comfortable 100% cotton t-shirt. Made from high quality combed cotton for smooth finish and comfort. NextWorks Tee is made with double needle stitching throughout the garment and sleeves for durability.
Jasper.ai Product Descriptions
- Sometimes it’s a party, sometimes it’s just the weekend. Quite frankly, you only need to buy one dress shirt for both occasions. Why clutter your closet with two separate shirts that take up way too much room? If you’re liberal in style and sensibility like us, this NextWorks t-shirt is perfect everywhere from work to play.
- Nextworks is great for summer. Your shirt will feel soft and comfy while getting the job done! Nextworks offers innovative patented technology that prevents odor, bacteria, moisture from building up under your armpits to keep you feeling fresh all day….in every way.
- This NextWorks shirt is so soft and nicely loose that it barely feels like you have anything on, but in the best way possible. It’s good for work or play because it pairs well with jeans or shorts at the same time. Grab one now and spread the word: a shirt that feels great has never been easier than when you shop with us.
The Verdict: Helpful
The AI tools were more helpful here than with the shorter content types. When each tool provided longer form content, it was easier to adjust on the fly, tweaking one way or another to a real, workable product description. Copy.ai did provide one nonsensical product description, however they all provided descriptions that were helpful.
- Copy.ai seemed to stumble of the friendly tone of voice item, directly including it as a product feature, rather than how I wanted the product descriptions to sound. The other two tools did not have trouble with this.
- Copysmith.ai provided good product descriptions of various lengths to work with.
- Jasper.ai also provided good product descriptions with an interesting variety of choices to work with.
So, what is our verdict? Copy.ai consistently underperformed our expectations of helpfulness. Copysmith.ai did better, providing content that was helpful and at least usable. Conversion.ai, while not perfect, consistently provided content that was better than the other two.
Five Things We Learned Using AI Content Tools
They are not quite exactly what they claim to be. While they claim that they can create human-quality content in a matter of seconds, we found that to be sort of true, and sort of not true. We did get some helpful content, but just as often the results were not helpful. The results were not quite human-quality, and it often needed us to re-write to be workable and usable for our applications.
For short content, it just didn’t help. In order for these products to work, you need to provide a keyword or two, and some sort of information or description for the tool to work from. When writing short content, such as meta descriptions, it takes longer to set the tool up that it does to just write the meta description. Unless you really need four or five options created, you can just create the short content on your own without this assistance. In other words, for short content, these tools don’t really save you much time.
They can help with writers block. This was probably the most useful part of the tool. When you are simply struggling to figure out how to say something, they can help. They can expand upon your content and show some options. While I found that I often didn’t always use the content it suggested, it did get the gears in my head turning again, so that I could start producing content. I would generally re-write much of what was suggested, but it would help get me inspired and keep me moving.
They can make you lazy, if you are not careful. You have to be careful not to rely on these tools too much. You still need to the the primary writer and source of content. If you lean on these tools too much, they will make you lazy. Be sure to read what they create, and adjust. Don’t slack off; don’t think these tools will magically create things for you.
They can not replace a good copywriter. Not yet anyway, and probably never will. As I mentioned, these tools can help you get past writers block, which is helpful and valuable. But a good copywriter does way more than just shuffle words and expand text. If you are a content creator, they can be helpful, in some cases very helpful, but do not rely on these tools instead of a copywriter.
So, How Much of this Article was Written by AI?
As I mentioned at the start of this article, I used all three of the copy tools to help write this article. So much of this article is actually written by AI? Maybe 20%. I re-wrote most suggestions they made. I primarily leaned on them to help get me past any writers block, which would inspire me to write the next chunk of content on my own. For that, they were helpful, for sure.
Are We going to Keep using These Content AI Tools?
After trying these three tools out, we are only keeping one — Jasper.ai. It was the most helpful to us, and helped keep our content moving in thoughtful ways. It wasn’t always perfect, but it was helpful. This article was written in the long form editor in the Conversion.ai tool.
However, we are still going to use our human copywriters for our general projects, though Conversion.ai could be helpful in situations when it comes to generating content. Particularly in the ability to help push past writers block, and providing options for content variations.