I enjoy reading web novels.
As I read many web novels, I noticed the good and bad points of each story. This made me want to write my own novel, but I have a full-time job and I'm from a science background, so I never got around to it.
Recently, seeing information about local LLMs posted here, I thought about installing the hermes agent and writing a story on a topic I've been thinking about.
However, running Qwen on my local 3090 was still disappointing in terms of writing. ㅎㅎ
So, I recently tried competing with 재미나이 3.5 Flash (Antigravity), Claude sonnet 3.6 (Claude Code), and Ollama Cloud GLM-5.1 (hermes) while also setting a concept and sketching out the overall picture.
For a while, I was excited to get help developing the world view, characters, and overall plot, but when I tried to write the detailed prologue and first chapter, I saw AI's fatal flaw.
It's that AI is really bad at pretending not to know something.
When it came to creating new settings, it was truly amazing, but when writing the prologue and first chapter where the latter half concept shouldn't appear yet, no matter how much I tried to refine it or force it, "What if 재미나이 made this mistake? Why don't you try it?" even with some teasing, it would end up making me laugh because the latter half setting would sneak in.
To be specific, there's something like 'name'. Even though they haven't met yet and haven't even exchanged names, the description already includes a name. Even when the protagonist doesn't know the other person's 'age' or 'occupation', related expressions suddenly appear.
From the last holiday until today, I've only been able to write the prologue and first chapter, and I realized that AI is really bad at 'pretending not to know something it shouldn't'.
And these days, I feel even more deeply how great writers who write smoothly and interestingly are. 😊
I wanted to share this fun experience with everyone, so I wrote this post!
By the way, these are my personal evaluations of the three AIs I used. (The one on the left felt better)
Setting Structuring and Conflict Analysis Claude sonnet 3.6 > (Surprisingly) GLM-5.1 >= 재미나이 3.5 Flash(high) (glm-5.1 felt more objective)
Final Manuscript Writing 재미나이 3.5 Flash(high) > Claude sonnet 3.6 > GLM-5.1
Manuscript Proofreading Claude sonnet 3.6 > GLM-5.1 > 재미나이 3.5 Flash(high)
Surprisingly, GLM-5.1 was quite good at Korean (with a few typos). Its logical processing was also excellent. However, it had a smaller context size, which limited its ability to handle large-scale settings.
I recently read an article about DeepSeek V4 permanently lowering its price and tried using the DeepSeek V4-Pro model for writing, but I was most disappointed. The response time was too long (in minutes), and it showed hallucinations (writing content not defined in the setting) that I didn't even see in glm-5.1.
