fix: preserve Unicode boundaries in source preview

This commit is contained in:
최민영 2026-08-26 09:54:02 +09:00
parent 4f5faf373a
commit ed2121be76
4 changed files with 32 additions and 3 deletions

View File

@ -80,9 +80,11 @@ class SourceContentClassifier(
text text
} }
val isCharTruncated = boundedByLines.length > maxRenderedCharacters val renderedCharacterCount = boundedByLines.codePointCount(0, boundedByLines.length)
val isCharTruncated = renderedCharacterCount > maxRenderedCharacters
val rendered = if (isCharTruncated) { val rendered = if (isCharTruncated) {
boundedByLines.substring(0, maxRenderedCharacters) val endIndex = boundedByLines.offsetByCodePoints(0, maxRenderedCharacters)
boundedByLines.substring(0, endIndex)
} else { } else {
boundedByLines boundedByLines
} }

View File

@ -245,6 +245,17 @@ class SourceContentClassifierTest {
assertEquals("abcde", text.renderedText) assertEquals("abcde", text.renderedText)
} }
@Test
fun characterLimitDoesNotSplitUnicodeSurrogatePair() {
val limited = SourceContentClassifier(maxInputBytes = 1_000, maxRenderedCharacters = 1, maxRenderedLines = 5)
val result = limited.classify(content("😀x")) as ContentClassification.Text
assertTrue(result.isTruncated)
assertEquals("😀", result.renderedText)
assertEquals(1, result.renderedText.codePointCount(0, result.renderedText.length))
}
// --- defensive immutability --- // --- defensive immutability ---
@Test @Test

View File

@ -99,3 +99,13 @@ This is the append-only implementation and handoff record shared by Codex and Cl
- Known limitations: No Android device/emulator was used, since this task has no Android framework dependency (pure JVM-testable Kotlin); static formatting/check task remains unconfigured for the module. - Known limitations: No Android device/emulator was used, since this task has no Android framework dependency (pure JVM-testable Kotlin); static formatting/check task remains unconfigured for the module.
- Next recommended task: Codex reviews this commit read-only per `docs/workstreams.md` cross-review rules; TASK-006 UI remains blocked until Codex freezes the TASK-004/005 contracts. - Next recommended task: Codex reviews this commit read-only per `docs/workstreams.md` cross-review rules; TASK-006 UI remains blocked until Codex freezes the TASK-004/005 contracts.
- Next owner: Codex, for review. - Next owner: Codex, for review.
## 2026-08-26 — TASK-006A Codex integration review
- Agent and role: Codex, integrator and reviewer
- Commit/diff: Claude commit `979729a`, integrated as `4f5faf3`, plus Codex Unicode-boundary fix
- Review findings and resolutions: One P2 was found: UTF-16 substring truncation could split an emoji or another supplementary character into a lone surrogate. Rendering now counts and truncates Unicode code points, with a focused emoji regression test.
- Checks and results: Focused classifier, full unit-test, debug-build, lint, diff, and sensitive-pattern gates are rerun after the fix.
- Known limitations: TASK-006 remains blocked on completion and freeze of the TASK-004/005 API and navigation contracts.
- Next recommended task: Codex completes the TASK-004 branch/tag/tree/file API remainder, then TASK-005 repository selection and branch browsing.
- Next owner: Codex.

View File

@ -1,6 +1,6 @@
# TASK-006A: Classify source content safely # TASK-006A: Classify source content safely
Status: implemented by Claude; pending Codex read-only review Status: integrated after Codex review and Unicode-boundary fix
Owner: Claude Owner: Claude
Reviewer: Codex Reviewer: Codex
@ -79,3 +79,9 @@ Known limitations:
- Static formatting/check task is not configured for this module (unchanged from TASK-001). - Static formatting/check task is not configured for this module (unchanged from TASK-001).
No open P0-P3 findings from self-review against the acceptance criteria. Requesting Codex read-only review per `docs/workstreams.md` cross-review rules; Claude stops writing after this handoff. No open P0-P3 findings from self-review against the acceptance criteria. Requesting Codex read-only review per `docs/workstreams.md` cross-review rules; Claude stops writing after this handoff.
## Codex review closure
- Integration commit: `4f5faf3` followed by a Codex-owned review fix.
- P2: character truncation used UTF-16 code-unit indices and could split a supplementary Unicode character into an invalid lone surrogate. Codex changed the rendering budget to Unicode code points and added an emoji boundary regression test.
- No other P0-P3 findings remained after source, test, scope, and contract review.