Generative AI For Industrial Content Development: a ChatGPT, Writer, and Jasper Showdown

Can AI tools help to develop content targeted to engineers and technical buyers? 

The landscape is exploding with generative AI tools for content marketing. But when it comes to creating B2B marketing content, especially pieces targeted to engineers and technical buyers, are these tools actually helpful? This is what TREW Marketing set out to find through research and writing experiments.

The episode kicked off with Morgan Norris, TREW Sr. Brand and Content Strategist, walking us through the variety of tools that aim to help marketers across content strategy, SEO, and content development. As we honed in specifically on content development, the team narrowed their focus to two primary tools, Writer and Jasper. They also explained instances when ChatGPT might helpful, and made mention of other content tools which you'll note in the resources section below.

Jamie Tokarz, TREW Sr. Content Manager, and Hannah Schulze, TREW Content Specialist, walked through their experiments using the tools and what they found most and least beneficial. On the whole, the scales weighed heavily towards the negative, but all involved saw some benefit and think there is a positive outlook for gains as the tools mature.

The best experiences centered upon headline generation and branding rule enforcement. The tools also provided a nice starting point to overcome "blank page syndrome." The worst experiences were around long-form technical content and any piece of content where you'd like to reinforce your brand messaging and SEO terminology in an authentic manner. 

The team put together an AI for B2B Content Generation guide based on their research and experiments to date, including before/AI-generated/writer-edited live content snapshots which really drive home their lessons learned. This guide will be continuously updated as TREW learns and experiments more. 

Resources

• An Industrial Marketer's Guide to Generative AI
• Writer
• Jasper
• ChatGPT
• Canva Magic Write
• Grammarly
• MarketMuse
• Dall-E

Transcript:

When it comes to B2B content development, the landscape has absolutely exploded with AI tools to help you do your job more effectively, or do they help? Well, that's what we're here to discuss today. I'm bringing in several members of the TREW Marketing team who have been experimenting with AI for content development specifically targeted towards technical audiences. We'll talk about ChatGPT, Writer, and JasperAI primarily, but there's also some other tools that we've woven into the conversation that we've tried or have on our radar. So out of today's episode, you'll learn where you should be looking, where you should expect to get the most benefit and the least benefit from these tools, and where we think this is all headed in the future. It wouldn't surprise me if we repeat this episode a few times a year because, wow, are things changing so rapidly. So hold on to your chairs. Let's do this.

Welcome to Content Marketing, Engineered. Your source for building trust and generating demand with technical content. Here is your host, Wendy Covey.

Hi and welcome to Content Marketing, Engineered. On each episode, I'll break down an industry trend, challenge or best practice in reaching technical audiences. You'll meet colleagues, friends, and clients of mine who will stop by to share their stories. And I hope that you leave each episode feeling inspired and ready to take action. Before we jump in, I'd like to give a brief shout out to my agency, TREW Marketing. TREW is a full service agency located in beautiful Austin, Texas, serving highly technical companies. For more information, visit trewmarketing.com. And now on with our podcast.

Hey everyone, and welcome to another episode of Content Marketing, Engineered. Today we'll be talking about AI for content development, a very hot topic right now. I'm joined by three members of TREW Marketing. Morgan Norris, Senior Brand and Content Strategist, Jamie Tokarz, Senior Content Manager, and Hannah Schulz, Content Specialist. We're going to have a great time today. Thank you so much for joining me.

We're happy to be here.

Awesome. And Valentine's Day has passed, but Morgan and I are sort of still channeling it with our attire today. So happy late Valentine's Day, everybody.

Yeah.

Well, let's start really high level and just talk about the current state of AI tools for content development. It is so hard to keep up with. It seems to be changing every day and really being on hyper speed. So, Morgan, tell me a little bit about that.

Yeah, so the current state is that it's different every day. There will be new news by the time you listen to this from when we're recording, even though it's days away. And so a few things that have happened recently that are kind of driving the development of AI tools is Google's helpful content update began kind of over the summer. And what is happening there is Google is favoring this kind of I would call it like genericized content that answers questions. And so companies are trying to figure out how to answer these questions to grab onto some search traffic. And an easy way to do that could be to plug that question into a robot writing tool. The AI companies don't like it when you say robot writing tools, but to plug it into an automated writing tool and have it produce some blurb of content that you can use kind of on your page. And so as those search tendencies continue to favor information like that, then you'll continue to see these tools kind of have relevance. So there's a few different issues that come up here and questions that this produces. Can I have an AI generator?

Write all of my web content? Can I have an AI generator? We've got a content plan that has one blog per week that we're going to post and one dated white paper a month. Could I just have a tool write those? That sounds really compelling and kind of man, if we could get this to work, that would be incredible. And so Jamie will talk in a little bit about we took on a process at TREW Marketing to evaluate and test some different tools. And so we've got a bunch of different kind of recommendations on what's working well now and what's not. But we're definitely in the thick of doing that and making sure we're staying on top of it. A couple of things to know. There's a close tie between some of these tools and SEO platforms. And as the content development AI tools are rising, so are different SEO platforms. We've all been used to kind of a few premium SEO platforms, but you're seeing other platforms come up now, like take a market news, for example, where you've got an SEO platform where they're suggesting terms and going as far as suggesting headlines and sub headlines.

And here's the kind of content that you need to include in your writing. And so those tools are starting to marry together a little bit. But what we're seeing is for technical audiences in particular, to have a platform that ties those two together is fairly cost prohibitive for most companies, unless you're maybe looking at a billion dollar plus company. And what's also happening is they're not yet great at capturing that long tail of really, really technical information that's important to technical audiences. And so this makes me think of five or six years ago, there was a big rise in content development firms. And what happened is companies moved away from creating SME based content and just handing over some keywords to a content development firm who would charge by the word and spit back some kind of genericized content. And that, for technical audiences, fell really flat really quickly. We had clients come to us and say, we tried this and it's not working. Our prospects don't trust this content. Our own SMEs don't want to put their names on it because it's not technical enough. And so anyway, it feels a little bit similar right now with these AI tools when we're looking at targeting technical audiences.

And so those are just a lot of the things, the news points, the kind of questions that we have and the issues that we're working through right now. And we'll talk more about recommendations in a minute, but just to kind of get us kicked off, that's what it looks like.

It's interesting, you talk about the tool landscape and the overlap between SEO tools that help inform content strategy, and then you have content development tools and that are oriented towards marketers and writers writing for that purpose for marketing. And then you have this overarching thing which are just content generation tools not oriented towards marketers that are kind of general purpose, like GPT. So with that, Jamie, what tools are you watching and how are they different?

Yeah, so we took a look at quite a few tools recently and then decided to perform a more thorough evaluation, specifically on what's called Co write from Writer and JasperAI. And the reason we chose these two tools is they are really geared towards marketers. They have content platforms that have different types of content you can select from to write. And then of course, we also looked at ChatGPT as kind of the hot tool being talked about right now that is really more of like a Q and A platform. You sort of question into it. ChatGPT generates your answer. So, not quite as geared towards marketing as some of the other tools we looked at. But we chose those two tools again, cowley and JasperAI, they seemed pretty robust in their offerings for the price lead. And interestingly, JasperAI also just announced an integration with HubSpot. So those two tools, we spent some time evaluating different offerings they had, such as their Headline Generator, their Generate a Blog Outline, generate a Blog post, and then even they have some Facebook ad writing tools that we looked at a little bit as well.

Okay, so you did a lot of testing.

I can't wait for you to talk about that.

I had a preview of this. There was a lot you guys really put these tools through the paces. So Hannah, what AI functionality did you find worked and was beneficial to you and your day to day job?

Yeah, a lot of times now folks are talking about these tools in terms of what they can do in the future and kind of the prospects of them. When we think about what we can do right now, we're having better luck with these higher level activities. So think, brainstorming, think. I'm a writer. Going from a blank page to creating a full document, perhaps coming up with some different sourcing questions. These are going to give you these higher level notes to say, okay, this is the place to start, but it won't necessarily get to the technical depth that we're always looking for. And we'll get into that more as we continue this conversation too.

So what did you guys find was not beneficial with these tools then?

Yeah, so in experimenting with these tools, so while they can generate a lot of content for a blog post, writing some of the more in depth pieces that we know our engineering audiences crave like white papers or case studies or act notes, there was not a good solution for that quite yet. And even with the shorter form content like blogs, we saw a lot of repetition throughout them. So for example, with using ChatGPT, we asked it what should be considered when designing a new test system and it gave us a list of ten points, but nine of those points started with this test system should be able to so there was a lot of repetition throughout. And even with some of the more marketing focused tools like Brighter Cowright and JasperAI, there was repetition between paragraphs that would start a new section and then sometimes it would repeat some of the content it already wrote and kind of to that same point. The writing is pretty basic. A lot of these tools are programmed to write at a 6th grade level and that is to make it simple so all audiences can understand.

And that results to us in a lot of not so much detail, just a lot of definition. And from the research we've done, we know that that's not necessarily what our technical audiences are looking for, they're looking for the detail. So we saw that that was lacking a bit and also not necessarily always pulling facts correctly, especially around what companies do or what products do and we'll touch on this more. But I think that that has to do with back to not wanting to pull things directly from online, but knowing that these tools are getting their information from the internet. Also, I think another thing that was AI generated at getting to that level of writing that most engineering audiences would expect, and this is a little speculative, but I think if it were to get to that point it might sound like reading a textbook. And as marketing professionals that seems kind of like opposite of what a lot of our audiences are looking for. We spend a lot of time making sure our customers have a distinct voice that's unique to their brand and I don't know that reading a textbook is what they're looking for right now.

And one more thing that I do want to touch on, on that kind of what's not working right now is I think from a marketing perspective that these tools are not set up at this point to take a customer, a potential customer and guide them through a purchasing journey. It's more one off content. So it's not what we're looking for as far as like, here's some introductory content. You're moving further down that purchasing funnel and getting closer to making a decision.

Okay, wow. That was kind of a long list where the tools are not today. And one of the ones that stuck out to me that you said was the tool is trying to give you unique content, but yet you still want it to talk in your brand voice and align with your company messaging, but it's avoiding that. So it's kind of this catch 22. So I guess that leads me to ask, what about Plagiarism? So original content versus just repeating something that's already out there? What are the concerns around that?

Yeah, I can talk to that one. So we really found Plagiarism not to be an issue. It's scouring already created content. So Plagiarism is not an issue. But because of that, what's happening is these tools are pulling content from websites and rewording them and I think sometimes rewording them, like sentence by sentence. And if you take a sentence about there's not really another word for a dynamometer, right? It is what it is. And when you get into these technical details, they can't be reworded or they become inaccurate. And so that's kind of an issue. That's like the double sided point of it, of their not being Plagiarism. The other thing is, if I put my branding hat on right, we create messaging for companies and it's like, man, this messaging has got to be everywhere. Your customers, your prospects only interface with you once or twice a month. Maybe they get your e newsletter and you've got to have your brand message has to be out there first and foremost, the descriptor that comes after your product or the benefits that your company provides or your tools provide. And if you're rewording those all the time, you're going to lose that brand awareness and that brand kind of iteration there.

And so I think that's really important. One thing we did, I had a blog post to write for a technical company and kind of put in my working headline and what my subsections were going to be, the things that we outlined normally when we do content planning for a client, right? So we kind of got that and then we're going to go interview an estimate and figure out kind of all the details that fit in here. And I put that in because I was like, man, what if this made my job so much easier? What if we could create three times as much content for clients? Or what if this was a cheaper process? So I put in all those inputs that we have and it generated all this content that ended up being I mean, I, I pulled 2000 words probably. It was a whole what would be like technical blog post, but it was very generic information, as if you kind of dumped like Wikipedia in there and it sounds like it's written at this lower level. I was just in my kids school and elementary school, and they're teaching them, first, do this, second do this, all those transitions.

And that's how these tools write, right? Which is it just feels not completely natural yet and they'll get there and get better. But that's one thing, just that brand mantra and messaging gets lost if you kind of give way completely to a tool like this.

Morgan when you did that experiment, did you have a similar issue that Jamie experienced where it would repeat the same phrases and first yes.

So I went back, actually, and I had this now this blog post was written and I kind of went through and highlighted and thought, well, I'll ask my estimate about this. Is this relevant? Is this relevant? And we kind of got to like, no, that's not what we're talking about. It's not relevant. I just side by side compare of what it produced and what I ultimately like a marked up version of what my final post looked like. And they it was probably 96% changed. I kept a couple of introduction sentences, maybe two introduction sentences and one definition little call out thing, but that was about it.

So it didn't exactly save you time.

It didn't save me any time, actually. And it was part of this experiment process we're trying to figure out and go through. But I will definitely say it did not even cross my mind to go use it again for the next project. It felt what ended up on my page was pretty distracting from a total content piece perspective. And like Hannah said, there's things that we can do. It is great to have there is a there's a point as a writer when you sit down at your desk and you're like, OK, brand new topic. Like, I understand the level of technicality that goes to, but this is maybe an application I haven't written about before or something like that, to have like a little brainstorm buddy. Like if you put in a headline and see what it spits out, just it's the same type of research I would do if I were going to research a topic. And I would go to electronic design and look at what articles they have about that topic and kind of get the lay of the land as to what's being talked about right now. It's kind of another tool for that purpose right now.

Hannah, did you find that benefit as well, that it was like your brainstorming buddy and avoiding that blank page syndrome?

Yeah, absolutely. It's interesting because we get these kind of new tools bombarding us in the news and it's exciting. And I continually have to kind of remind myself about our audience and center that we have these writing skills we're pulling from these great content tools, but we're also saying, hey, we have an engineering audience and we need to focus on the things that they need. And luckily, we have research that we can point to to kind of feel grounded and close to the audience. And one of the points I wanted to bring up today was that 43% of engineers will filter through at least five pages of search results to see what they're looking for and defined in answer to their question. Only 4% stop at that first page. So it's also not worth it for us to kind of stop at that first. Whatever the AI tool gives us, we have to go deeper, and we have to look at what's coming out and really assess that and make it to be kind of richer and more fulfilling, knowing that Page One isn't necessarily the end all, be all.

Yeah.

And the idea of making your content less relevant or appropriate to chase Page One, shooting yourself in the foot because that behavior says they're not Page One to make it quality content right. That's relevant and compelling.

And we know that engineers trust content that's written by other engineers, or 2023. Research also shows us that 64% of respondents consider engineering experts as vendor companies to be extremely or very trustworthy. And if we're seeing, just like Morgan said, elementary middle school writing, we're seeing technical inaccuracies or maybe data that's not representing current industry insights. We're not really representing these areas where technology is evolving so rapidly.

Yeah. And of course, the risk of losing trust by having an engineer's name on content that's substandard like you just described.

Absolutely. And it's worth it to look into these tools. And my colleagues will continue to talk about how we're going to do that going forward, but we also need to really protect the trust that we've worked hard to build over time.

Okay, so how can we use the tool today? Should we be using the tool today? What do you guys think?

Yeah, I'll jump in since I spend a lot of time talking about the gaps in the tool. But there are some good use cases for AI writing tools for content development, specifically for technical audiences. So back to where Morgan started. Talking about this feels very similar to trying to grab those Page One rankings. Sometimes that is something some companies really want to do for a very specific term, and something we've seen emerging recently and you may not be familiar with them, are what's called a glossary page. And we're seeing it a lot now where there might be a specific term. For example, if you were to search WiFi Six, the first result will definitely be a glossary page from intel. And this is a great tool for that because that page is what is WiFi Six, and it defines what that is. And that was the goal of that page. It's not to go deep into WiFi Six. It's to give a very basic definition of what it is and get people to the intel site and show that intel is a leader in WiFi Six and that technology. So glossary pages I think would be very helpful for developing those pages and similarly, anything that's repetitive content.

So like product descriptions for a large family of products where there's very small differences between them. I could see a tool, an AI tool, being really helpful for that right now, and even, like, introductory content to a longer technical piece if there's a less technical introduction into a topic that you need a little bit of assistance with getting that started. I think that these tools are in a good position for that type of content as well.

Great.

We've talked about the idea of it as a brainstorming tool, for one, but also realize that these tools have builtin structures that you can leverage. And while it may not be a perfect fit just yet, as we see these tools evolve, SMEs might have better support tools to be able to come up with outlines or put their thoughts down to prepare to speak with us.

A cool feature too. Co write has a tool in there, like, so think of like, grammarly, but you can put all of your company parameters in there and it will check your style as you go through. As you're creating content, it'll say, typically your tone is more like this, but this is sounding this way. Here some suggestions to clean that up and that again, it's an expensive tool and you'd have to be really committed to using it across the whole company. But I had one of the marketers who took the Content Writing Engineer course last year. I saw Content Marketing World, and we had just seen this demo where Writer was talking about Co write used in this way, and he said he's in charge. His company keeps acquiring companies and he keeps acquiring all these one off marketers and all of their content. And he said, I think instead of hiring a person to go through and kind of flag all the errors not even errors, but discrepancies in tone and make all this stuff sound the same, he was like, I think I might just try this tool and see if that helps me bring everything under one brand and get all these marketers that I've acquired kind of on the same page, which I thought was a really great use of that technology for kind of a cohesion across the brand.

So maybe a rebrand or acquisitions like that, it could be interesting.

Yeah, I remember when you guys when we talked about this subject before headline writing came up as another beneficial feature.

Yeah, they have. Co write has a headline generation tool where you can kind of put your topics in. A lot of times when we give clients content, we like to give them a couple of options of headlines. You kind of put in a topic and it will spit out like ten different options, which again, as like a brainstorm tool. Just kind of a helpful starting point at times, right? I don't have to think of all ten of those, because whenever you're writing the writing process, right, you draft those and you go back and pick at them and edit them and stuff, and it kind of gives you a starting point. You'll cross out, you'll just throw away five immediately, but then there's five there that you can kind of iterate on. So that's a tool for sure.

Okay, little Lukewarm to positive in your Smith headline. Got it. Okay, so how should content marketers think of AI moving forward? Let's take out a crystal ball.

Yeah. Where do we think these tools are.

Going and how are they changing and how do people keep on top of it?

So I think they'll get better and better. I think they're built to be learning tools, and as more people use them, they will grow to be better and stronger. I think that some of the issues that we're seeing with maybe tone and repetition and kind of like, level of writing will get worked out. I've seen one demo where you could tell something to write at a certain level, like have it right at a fourth grade level and put some errors in there so it actually looks like a fourth grader wrote it.

Oh, gosh. Your kids, Morgan.

That would be the go home and read The Great Gatsby and write an essay on the primary theme and site references. That's over. That's not happening anymore in school because of tools like this. And so we've got to find other ways to kind of test knowledge. And in a way, the content that we create for our clients tests knowledge in the same way we're trying to create content that somebody will read and go, oh, my gosh, this company, they're engineers, they know what I'm going through and they know how to fix it. And that's still out of reach, I think, for these tools. But a lot of the tools have free trials. You can hop in, you can use them. ChatGPT, you can just it's not even like a trial. I think the whole thing is like just free to use. They have a paid platform, I think, but you can hop in and just ask it some questions. I didn't experiment that way. It was great. Ask it some marketing questions, asked it some engineering related questions, and it was just interesting to see the type of content that came out. Again, kind of repetitive and stuff, but kind of a fun tool to play with.

One interesting thing about ChatGPT, and they're very upfront about this, though, is that it stopped pulling data in 2021. And so the intro screen of ChatGPT says, just a warning, this information could be harmful, it could be inaccurate, and it could be biased. And so nothing's like, perfect. So that's all to say. It's not a miracle, and you aren't going to be able to sit on the beach all day and have an AI tool write all your content, but it's just not we're not there yet. A few things that I'm interested in looking at as time goes on are if Google starts penalizing bounce rate, so starts penalizing bounce rate more, I guess because what's happening is when we're writing to answer these generic questions, what's going to happen is you're asking Google a question or you're searching for something and you hop over and get that answer within a few seconds and you hop right back off. That's not necessarily a win to a company, right? That metric is not going to feel great. Yeah, people are coming, but they're not staying or engaging with anything that we're doing. And so I'll be interested to see how that plays in.

And then I'm also interested for some AI company to be able to reverse the process and create derivative content. So I would like all the AI companies, I would like to put in a white paper that we've written and have it create a three series blog post. Again, that's not something that takes a ton of time, but it kind of would be a nice click and go feature. Like write the intro for this and then split it up and then write the CTA part at the bottom. Or here's a white paper. Write the landing page content, things like that, where we've already done the legwork on the content. So I'm watching, we're waiting for that. One thing we're committing to do is evaluating these tools on every other month, every quarter basis, and just continue updating our team and our clients on what's working, what's not, what are ways that we should engage. And so that's something we're doing. There will be an AI guide on TREW Marketing site under Resources that again will continue to update. We're going to show some of these things that we learned through that kind of experiment. Here's what we inputted, here's what came out, here's what we did with it.

And so you guys can go there and just kind of trust that that will be continually updated as we learn new things and engage with new platforms and stuff.

If it's not confidential, I think it would be beneficial just to see those examples that you guys described where you have that repetition or all the red lining to kind of drive the point home. Very good. So the guide will be used at the Resources section of the TREW Marketing website and then Jamie, Hannah, any other parting thoughts? Anything else we didn't cover today?

One thing to add on to what Morgan just said about how we're continuing to evaluate tools is that it's really important because new tools are emerging like really fast and they're coming with tools that we're already using. So like, for example, Canva just came out with Magic great. And it's more like ChatGPT in. That you ask questions and it can answer them. It's like basically performing the research or Google search for you directly from canvas. So we're seeing more and more places where these types of tools are emerging. But as that's all happening too, I think that the platforms that are publishing this content, the search tools, are also getting a little smarter. And I've already seen some indications that there's going to be tags added to the content that kind of shows like this was pulled from an AI tool. I've only read articles about that happening, and I don't know exactly how that's going to be happening, but I think that's one thing too is that not only will those tags be there, but I also think search engines are very smart. Google is very smart. And to Morgan's point about penalizing for bounce rates on these types of pages, like AI generated pages, could that be another thing that is going to be penalized by search in the future?

And that's something we're definitely going to be watching too.

Yeah, I got an email. It was like B to C type of content. Not B2B, but the email at the bottom of the email said this was completely generated with an AI tool, which was just interesting. It was like an email about events happening in the area type of thing. So it's interesting to start to see those things come up and the acknowledgment.

That it feels like a little bit of a hedge is sort of off. You're wasting my time if you're not fixing those problems. I don't know.

Yeah, it's going to be great to watch and keep up with. And just like Jamie said, it's going to get smarter and better and will create new efficiencies and also new things to work on, I think.

Just looking ahead, I'm interested to see what we can glean between the changes with Google search and the emergence of all these AI tools. Research will come around again. And so as we think about what's happened to this year and what's going forward into 2024, 2025, these will be questions that we're going to want to ask engineers.

Yes. Oh, yes. We need to start brainstorming on those questions.

We didn't mention we didn't mention imagery, but there are also imagery tools. Not we're not there yet for this, like, really technical stuff because a lot of times when we're asking for images from our clients, they're like images for a case study of an actual application or it's a diagram of something, but fun to play around with. If you're looking, you can insert into an AI tool. Like you want a dog riding a horse seating, a cheeseburger jumping over the moon, and you can get it. Again, very helpful for a case study. Not saving you any time over here, but still a ball.

Yeah, they certainly will. Thank you all for being here today. I will include links and names of all the tools we discussed today so that people can find them. And thank you for your commitment to revisit this every quarter. So maybe we'll as well, maybe twice a year at least, if not more, will put together this panel again, come back on the podcast as things change and do an update. I think that would be great, if you guys are willing.

Yeah, that'd be great.

Absolutely. Great. Well, thank you for being here.

Thank you for having me.

Take care.

Thanks for joining me today on Content Marketing, Engineered. For show notes, including links to resources, visit trewmarketing.com/podcast. While there, you can subscribe to our blog and our newsletter and order a copy of my book, Content Marketing, Engineered. Also, I would love your reviews on this podcast, so please, when you're get a chance, subscribe and leave me your review on your favorite podcast subscription platform. Thanks and have a great day.