Hello
I made a harness for Vibe coding using Vibe coding. It took about a month to make. At first, I kept making mistakes, but after July 16th, when the Grok Build source code was released, my skills improved significantly (I had an AI analyze it and create a framework). Initially, I only wanted to refer to the source code, but I liked Grok Build so much that I switched to using it as a base.
I'm curious how other people make harnesses. Anyway, I'm happy because I achieved my initial goal and visible results. I want to share the development content so far. I hope it will be helpful to someone.
Goal: Make Qwen 27B Usable
I was using existing open source code to make a harness, but the performance wasn't as good as I expected. The source code had various functions, but the most important LLM orchestration function was weak. Since sub-agent context maintenance wasn't working, it felt like each agent was playing on its own, which reduced its usefulness. Of course, I really like the plan mode.
When I tried Codex... this amplifies noise. The functions are powerful, but you need a persistent LLM to use them effectively. Codex 5.6 sol and Luna are also like that, and ChatGPT models are fundamentally persistent. Models like Qwen, which are impulsive, often get stuck in noise and loops.
Claude code still produced good results, but there was a lot of code that didn't work. When I looked into it, the surface seemed fine, but there were a lot of shoddy constructions (but at least it was something). To fix these shoddy constructions, I had to use high-level APIs, which wasn't very token-efficient and felt ambiguous. Fundamentally, using Claude doesn't make a dumb model smart, so the final touches were difficult despite using a lot of tokens.
Features of Grok Build
It's fast. And the quality of most code is better than open source (this will be shown in the table below). As a latecomer, the development period was short, so it feels like the basic functions are only filled for each function, but the orchestration is very strong, probably because Grok Heavy has been used before.
Disadvantages: The permission settings seem a bit weak. For example, assigning permissions to sub-agents like in open source doesn't seem possible, and it seems that permissions are handled integrally (so if there is a leak somewhere by mistake, everything is exposed?).
But here's the thing. (Important) Grok Build has one hidden feature. It even has a native vector memory function built in that Grok itself doesn't use (it says experimental in the documentation). If the user adds an appropriate vector embedding model, they can not only search by keywords but also maintain context through semantic vector search. Various options such as time decay, MMR, and boosting are all available. I was able to easily implement the vector-based context maintenance function I wanted to create.
I don't understand why they hid this good feature. Grok only uses keyword search for its vector memory function (why?). Claude code is the same way. Of course, you can extend it with an external MCP, which is a shame. It's probably just an experimental feature.
So far, I have been using the vector loop algorithm (?) that I painstakingly created by modifying open source code. The structure made sense, but I was struggling with how to implement things like weights and MMR... Using Grok Build's powerful orchestration + memory function worked out great. As expected, enterprise products are of a different quality. It only took two days to transcend the vector-based loop I had created before.
Codex CLI and VectorBuild 1.1 Code Result Comparison
Five questions from Terminal-Bench 2 were evaluated using the same gpt-5.6-luna, reasoning high.
Official Test Scoring
Issue | Native Codex CLI | VectorBuild 1.1 Vector |
|---|
Asynchronous task cancellation/SIGINT handling | ❌ | ✅ |
LLM inference batch scheduler | ✅ | ✅ |
Video takeoff/landing frame detection | ✅ | ❌ |
Custom memory heap collision | ✅ | ✅ |
Proprietary format compressed file generation | ❌ | ✅ |
Final Score | 3/5, 60 points | 4/5, 80 points |
Code Quality Assessment
문제 | Native 코드 | VectorBuild 코드 |
|---|
비동기 작업 취소 | 7/10 — 구현 방향은 적절하지만 대기 중 작업과 SIGINT 정리가 완전하지 않음 | 8/10 — 자식 예외 감지와 실행·대기 작업 정리를 모두 구현해 더 견고함 |
배칭 스케줄러 | 7/10 — 동적 계획법으로 요구 조건을 충족하지만 일부 내부 구현 의존성이 있음 | 7/10 — 테스트는 통과하지만 공유 shape 값 하드코딩으로 일반성이 낮음 |
영상 프레임 검출 | 8/10 — 비공개 영상까지 통과했고 노이즈 대응도 안정적임 | 7/10 — 분석 구조는 좋지만 고정 임계값 때문에 비공개 영상에서 1프레임 오차 발생 |
힙 충돌 수정 | 9/10 — 작고 정확한 수정으로 문제를 해결함 | 9/10 — 필요한 파일만 최소 수정한 간결하고 정확한 해법 |
압축 파일 생성 | 4/10 — 제한 안에 검증 가능한 결과물을 완성하지 못함 | 8/10 — 2,277바이트 결과를 생성했고 디코딩 결과가 원본과 완전히 일치함 |
코드 품질 합계 | 35/50, 70점 | 39/50, 78점 |
최종 결과
| 공식 테스트 | 60점 | 80점 |
| 코드 품질 | 70점 | 78점 |
VectorBuild가 공식 테스트와 코드 품질 모두 앞섰습니다. Native는 영상 처리 코드의 일반화가 더 좋았고, VectorBuild는 비동기 정리와 압축 파일 생성처럼 다단계 검증이 필요한 문제에서 더 완성도 높은 결과를 냈습니다.
결론 : 토큰 2배 시간도 1.5배
하지만 조금 더 똑똑해짐
차이는 맥락 유지입니다. 경량 모델이어도 일단 코드는 잘 만듭니다. 하지만 맥락 유지가 안되면 코드 작성중에 삽질을 하고 전체 코드베이스가 스파게티처럼 꼬이게 되죠. 벡터 기반 메모리는 이런 머리나쁜(?) Ai에 암기력을 심어줍니다.
어쨌든 전 오픈코드나 코덱스로는 만들지 못하던 프로그램을 그록 빌드로는 만들 수 있었고. 이에 기분이 좋습니다. 이토록 좋은 프로그램을 공개로 풀다니 spacexai 에 감사합니다.
모두 즐 점심 되세요
