fix: preserve Unicode boundaries in source preview
This commit is contained in:
parent
4f5faf373a
commit
ed2121be76
@ -80,9 +80,11 @@ class SourceContentClassifier(
|
||||
text
|
||||
}
|
||||
|
||||
val isCharTruncated = boundedByLines.length > maxRenderedCharacters
|
||||
val renderedCharacterCount = boundedByLines.codePointCount(0, boundedByLines.length)
|
||||
val isCharTruncated = renderedCharacterCount > maxRenderedCharacters
|
||||
val rendered = if (isCharTruncated) {
|
||||
boundedByLines.substring(0, maxRenderedCharacters)
|
||||
val endIndex = boundedByLines.offsetByCodePoints(0, maxRenderedCharacters)
|
||||
boundedByLines.substring(0, endIndex)
|
||||
} else {
|
||||
boundedByLines
|
||||
}
|
||||
|
||||
@ -245,6 +245,17 @@ class SourceContentClassifierTest {
|
||||
assertEquals("abcde", text.renderedText)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun characterLimitDoesNotSplitUnicodeSurrogatePair() {
|
||||
val limited = SourceContentClassifier(maxInputBytes = 1_000, maxRenderedCharacters = 1, maxRenderedLines = 5)
|
||||
|
||||
val result = limited.classify(content("😀x")) as ContentClassification.Text
|
||||
|
||||
assertTrue(result.isTruncated)
|
||||
assertEquals("😀", result.renderedText)
|
||||
assertEquals(1, result.renderedText.codePointCount(0, result.renderedText.length))
|
||||
}
|
||||
|
||||
// --- defensive immutability ---
|
||||
|
||||
@Test
|
||||
|
||||
@ -99,3 +99,13 @@ This is the append-only implementation and handoff record shared by Codex and Cl
|
||||
- Known limitations: No Android device/emulator was used, since this task has no Android framework dependency (pure JVM-testable Kotlin); static formatting/check task remains unconfigured for the module.
|
||||
- Next recommended task: Codex reviews this commit read-only per `docs/workstreams.md` cross-review rules; TASK-006 UI remains blocked until Codex freezes the TASK-004/005 contracts.
|
||||
- Next owner: Codex, for review.
|
||||
|
||||
## 2026-08-26 — TASK-006A Codex integration review
|
||||
|
||||
- Agent and role: Codex, integrator and reviewer
|
||||
- Commit/diff: Claude commit `979729a`, integrated as `4f5faf3`, plus Codex Unicode-boundary fix
|
||||
- Review findings and resolutions: One P2 was found: UTF-16 substring truncation could split an emoji or another supplementary character into a lone surrogate. Rendering now counts and truncates Unicode code points, with a focused emoji regression test.
|
||||
- Checks and results: Focused classifier, full unit-test, debug-build, lint, diff, and sensitive-pattern gates are rerun after the fix.
|
||||
- Known limitations: TASK-006 remains blocked on completion and freeze of the TASK-004/005 API and navigation contracts.
|
||||
- Next recommended task: Codex completes the TASK-004 branch/tag/tree/file API remainder, then TASK-005 repository selection and branch browsing.
|
||||
- Next owner: Codex.
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# TASK-006A: Classify source content safely
|
||||
|
||||
Status: implemented by Claude; pending Codex read-only review
|
||||
Status: integrated after Codex review and Unicode-boundary fix
|
||||
Owner: Claude
|
||||
Reviewer: Codex
|
||||
|
||||
@ -79,3 +79,9 @@ Known limitations:
|
||||
- Static formatting/check task is not configured for this module (unchanged from TASK-001).
|
||||
|
||||
No open P0-P3 findings from self-review against the acceptance criteria. Requesting Codex read-only review per `docs/workstreams.md` cross-review rules; Claude stops writing after this handoff.
|
||||
|
||||
## Codex review closure
|
||||
|
||||
- Integration commit: `4f5faf3` followed by a Codex-owned review fix.
|
||||
- P2: character truncation used UTF-16 code-unit indices and could split a supplementary Unicode character into an invalid lone surrogate. Codex changed the rendering budget to Unicode code points and added an emoji boundary regression test.
|
||||
- No other P0-P3 findings remained after source, test, scope, and contract review.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user