Motif and ExaOne opened a chat for 14 days, so I just tried it. Both are very disappointing. My tax money ㅜ.ㅜ
For example, if I have to arrive at station A by 9 am this Saturday morning. What time should I take the train from station B? When I ask this question,
ExaOne is very, very cautious or doesn't give an answer for a long time. Is it because the server is slow or because it thinks deeply about the process, but the thinking process itself seems empty.
Motif is fast. It says that it takes 51 minutes from A to B, so you should take the 8:09 train. Of course, this isn't the answer I expected.
If I ask the same question to ChatGPT, it will find the time of the train in that direction based on the weekend subway timetable and tell me accurately.
In addition, I asked about a technical paper, and both gave fairly good answers. However, if ChatGPT feels like a doctoral student or professor, this feels like an answer from a master's degree freshman. I wonder if they really understood it. ExaOne is too slow (is it because the server is slow?). The content is also...
Motif even misinterpreted the abbreviation and explained it incorrectly. Even when asked again, it doesn't realize that it's wrong. Only after I correctly spelled out the abbreviation did it acknowledge that it was right.
It seems to be used in government offices or public institutions where foreign models cannot be used, but there is still a long way to go. I haven't tried other models, so I don't know. In my experience, it seems to be about two years behind (of course, the benchmark score is good).
▶ Original source: https://chat.motiftech.io/chat
▶ Original source: https://k.exaone.ai/
