-
Notifications
You must be signed in to change notification settings - Fork 0
[NOOK-191] 게시물 상세 위치 기반 장소 매칭 정확도 개선 #124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
c72d272
[NOOK-191] 게시물 상세 위치 기반 장소 매칭 개선
dh1010a 2a6b3d9
[NOOK-191] 상세 주소 후보 매칭 보완
dh1010a be65513
[NOOK-191] 상세 위치 미확인 장소 연결 차단
dh1010a fe95c0c
[NOOK-191] 원문 상호 근거 매칭 보완
dh1010a 8e243ad
[NOOK-191] 장소 검색에 상호명 쿼리 보장
dh1010a 82eb44f
[NOOK-191] 장소 검색 재현 검증 보완
dh1010a File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # NOOK-191 게시물 상세 위치 정보 보존 및 장소 매칭 정확도 개선 | ||
|
|
||
| ## 목적 | ||
|
|
||
| Instagram 게시물 원문이나 이미지에 명시된 층·호 정보가 장소 단서 추출과 후보 선택 과정에서 | ||
| 유실되지 않게 하고, 같은 건물 내 다른 매장이 연결되는 문제를 수정합니다. | ||
|
|
||
| ## 범위 | ||
|
|
||
| - 본문·해시태그·Instagram 장소 태그·이미지 전사에서 명시적인 상세 주소를 장소 단서로 보존합니다. | ||
| - `1층`, `4층`, `B1`, `지하 1층`, `201호` 등의 상세 위치를 검색어와 후보 선택 근거에 전달합니다. | ||
| - 도로명·건물 번호 또는 층·호가 충돌하는 후보를 자동 확정하지 않습니다. | ||
| - 같은 주소의 다른 상호는 주소 일치만으로 연결하지 않습니다. | ||
| - 재현 게시물과 유사한 주소·층·호 조합의 회귀 테스트를 추가합니다. | ||
|
|
||
| ## 제외 범위 | ||
|
|
||
| - 지도 provider 교체 | ||
| - 장소·게시물 공개 API 응답 필드 변경 | ||
| - 장소 테이블의 주소 체계 또는 DB 스키마 변경 | ||
| - 기존 저장 게시물 전체 일괄 재파싱 | ||
|
|
||
| ## 성공 기준 | ||
|
|
||
| - 명시적인 주소와 층·호 정보가 `PlaceClue.addressHint`에 원문 그대로 보존됩니다. | ||
| - 상세 주소가 포함된 검색어가 일반 검색어보다 먼저 사용됩니다. | ||
| - 같은 건물의 다른 상호와 다른 도로명·건물 번호 후보가 자동 연결되지 않습니다. | ||
| - provider 주소에서 상세 위치가 생략되어도 상호와 기본 주소가 일치하면 정상 연결됩니다. | ||
| - 기존에 저장된 장소 단서 JSON을 계속 읽을 수 있습니다. | ||
| - `./gradlew check`가 통과합니다. |
95 changes: 95 additions & 0 deletions
95
...i-application/src/main/kotlin/org/every/nook/api/application/place/PlaceAddressMatcher.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| package org.every.nook.api.application.place | ||
|
|
||
| internal object PlaceAddressMatcher { | ||
| fun isCompatible(addressHint: String?, candidateAddress: String): Boolean { | ||
| val hint = addressHint?.trim()?.takeIf(String::isNotEmpty) ?: return true | ||
| val hintAddressKeys = addressKeys(hint) | ||
| val candidateAddressKeys = addressKeys(candidateAddress) | ||
| if ( | ||
| hintAddressKeys.isNotEmpty() && | ||
| candidateAddressKeys.isNotEmpty() && | ||
| hintAddressKeys.intersect(candidateAddressKeys).isEmpty() | ||
| ) { | ||
| return false | ||
| } | ||
|
|
||
| val hintDetails = locationDetails(hint).groupBy(LocationDetail::type) | ||
| val candidateDetails = locationDetails(candidateAddress).groupBy(LocationDetail::type) | ||
| return hintDetails.all { (type, details) -> | ||
| val candidateValues = candidateDetails[type]?.mapTo(mutableSetOf(), LocationDetail::value) | ||
| ?: return@all true | ||
| details.any { it.value in candidateValues } | ||
| } | ||
| } | ||
|
|
||
| fun addressKeys(value: String): Set<String> = BASE_ADDRESS_PATTERN.findAll(value).flatMap { match -> | ||
| val roadName = match.groupValues[1].groundingKey() | ||
| val buildingNumber = match.groupValues[2].groundingKey() | ||
| val buildingNumbers = buildSet { | ||
| add(buildingNumber) | ||
| if (buildingNumber.length >= MIN_COMPACT_BUILDING_FLOOR_LENGTH && match.isFollowedByFloorSuffix(value)) { | ||
| add(buildingNumber.dropLast(1)) | ||
| } | ||
| } | ||
| buildingNumbers.asSequence().flatMap { number -> | ||
| sequenceOf( | ||
| roadName + number, | ||
| roadName.removeSuffix(ROAD_SUFFIX) + number, | ||
| ) | ||
| } | ||
| }.toSet() | ||
|
|
||
| fun hasLocationDetail(value: String?): Boolean = value != null && locationDetails(value).isNotEmpty() | ||
|
|
||
| private fun locationDetails(value: String): Set<LocationDetail> { | ||
| val basementDetails = BASEMENT_PATTERN.findAll(value).map { match -> | ||
| LocationDetail(LocationDetailType.FLOOR, "-${match.firstCapturedValue()}") | ||
| }.toSet() | ||
| val aboveGroundSource = BASEMENT_PATTERN.replace(value, " ") | ||
| val floorDetails = FLOOR_PATTERN.findAll(aboveGroundSource).map { match -> | ||
| LocationDetail(LocationDetailType.FLOOR, match.groupValues[1]) | ||
| } | ||
| val compactFloorDetails = COMPACT_BUILDING_FLOOR_PATTERN.findAll(aboveGroundSource).map { match -> | ||
| LocationDetail(LocationDetailType.FLOOR, match.groupValues[1]) | ||
| } | ||
| val roomDetails = ROOM_PATTERN.findAll(value).map { match -> | ||
| LocationDetail(LocationDetailType.ROOM, match.groupValues[1]) | ||
| } | ||
| return basementDetails + floorDetails + compactFloorDetails + roomDetails | ||
| } | ||
|
|
||
| private fun MatchResult.isFollowedByFloorSuffix(source: String): Boolean = | ||
| source.substring(range.last + 1).trimStart().startsWith(FLOOR_SUFFIX) | ||
|
|
||
| private fun MatchResult.firstCapturedValue(): String = groupValues.drop(1).first(String::isNotEmpty) | ||
|
|
||
| private fun String.groundingKey(): String = lowercase().filter(Char::isLetterOrDigit) | ||
|
|
||
| private enum class LocationDetailType { | ||
| FLOOR, | ||
| ROOM, | ||
| } | ||
|
|
||
| private data class LocationDetail(val type: LocationDetailType, val value: String) | ||
|
|
||
| private val BASE_ADDRESS_PATTERN = Regex( | ||
| "([가-힣A-Za-z]+(?:대로|로|길)(?:\\d+[가-힣]?(?:길)?)?|" + | ||
| "[가-힣A-Za-z0-9]+(?:동|읍|면|리))\\s+(\\d+(?:-\\d+)?)", | ||
| ) | ||
| private val BASEMENT_PATTERN = Regex( | ||
| "(?i)(?:\\bB\\s*-?\\s*(\\d+)|지(?:하)?\\s*(\\d+)\\s*층)", | ||
| ) | ||
| private val FLOOR_PATTERN = Regex( | ||
| "(?i)(?<![-\\d])([1-9]\\d?)\\s*(?:층|F)(?=$|[^가-힣A-Za-z0-9])", | ||
| ) | ||
| private val COMPACT_BUILDING_FLOOR_PATTERN = Regex( | ||
| "(?<!\\d)\\d{2,}([1-9])\\s*층(?=$|[^가-힣A-Za-z0-9])", | ||
| ) | ||
| private val ROOM_PATTERN = Regex( | ||
| "(?<![-\\d])([1-9]\\d{0,3})\\s*호(?=$|[^가-힣A-Za-z0-9])", | ||
| ) | ||
|
|
||
| private const val ROAD_SUFFIX = "길" | ||
| private const val FLOOR_SUFFIX = "층" | ||
| private const val MIN_COMPACT_BUILDING_FLOOR_LENGTH = 3 | ||
| } | ||
27 changes: 27 additions & 0 deletions
27
...cation/src/main/kotlin/org/every/nook/api/application/place/PlaceCandidateDeduplicator.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| package org.every.nook.api.application.place | ||
|
|
||
| internal fun List<PlaceCandidate>.distinctLogicalPlaces(): List<PlaceCandidate> = | ||
| fold(mutableListOf()) { places, candidate -> | ||
| val samePlaceIndex = places.indexOfFirst { existing -> existing.isSameLogicalPlace(candidate) } | ||
| when { | ||
| samePlaceIndex < 0 -> places += candidate | ||
| candidate.isMoreDetailedThan(places[samePlaceIndex]) -> places[samePlaceIndex] = candidate | ||
| } | ||
| places | ||
| } | ||
|
|
||
| private fun PlaceCandidate.isSameLogicalPlace(other: PlaceCandidate): Boolean { | ||
| if (name.groundingKey() != other.name.groundingKey()) { | ||
| return false | ||
| } | ||
| val addressKeys = PlaceAddressMatcher.addressKeys(address) | ||
| val otherAddressKeys = PlaceAddressMatcher.addressKeys(other.address) | ||
| return addressKeys.isNotEmpty() && | ||
| otherAddressKeys.isNotEmpty() && | ||
| addressKeys.intersect(otherAddressKeys).isNotEmpty() | ||
| } | ||
|
|
||
| private fun PlaceCandidate.isMoreDetailedThan(other: PlaceCandidate): Boolean = | ||
| PlaceAddressMatcher.hasLocationDetail(address) && !PlaceAddressMatcher.hasLocationDetail(other.address) | ||
|
|
||
| private fun String.groundingKey(): String = lowercase().filter(Char::isLetterOrDigit) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
74 changes: 74 additions & 0 deletions
74
...ication/src/main/kotlin/org/every/nook/api/application/place/PlaceClueCandidateMatcher.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| package org.every.nook.api.application.place | ||
|
|
||
| internal fun Collection<PlaceCandidateSelector.Candidate>.compatibleWith( | ||
| clue: PlaceClue, | ||
| ): List<PlaceCandidateSelector.Candidate> { | ||
| val addressHint = clue.addressHint?.trim()?.takeIf(String::isNotEmpty) ?: return toList() | ||
| return filter { candidate -> PlaceAddressMatcher.isCompatible(addressHint, candidate.place.address) } | ||
| } | ||
|
|
||
| internal fun PlaceClue.isSupportedBy(candidate: PlaceCandidate): Boolean { | ||
| val explicitAddressHint = addressHint?.trim()?.takeIf(String::isNotEmpty) | ||
| if (explicitAddressHint != null && !PlaceAddressMatcher.isCompatible(explicitAddressHint, candidate.address)) { | ||
| return false | ||
| } | ||
|
|
||
| val hasCompatibleIdentity = hasCompatibleName(candidate) || hasNameEvidence(candidate) | ||
| if (explicitAddressHint != null && !hasCompatibleIdentity) { | ||
| return false | ||
| } | ||
| return evidence.isEmpty() || hasCompatibleIdentity || hasCompatibleEvidence(candidate) | ||
| } | ||
|
|
||
| internal fun Collection<PlaceCandidateSelector.Candidate>.descriptions(limit: Int): List<String> = | ||
| take(limit).map { candidate -> "${candidate.place.name}|${candidate.place.address}" } | ||
|
|
||
| private fun PlaceClue.hasCompatibleName(candidate: PlaceCandidate): Boolean { | ||
| val candidateName = candidate.name.groundingKey() | ||
| val candidateAddress = candidate.address.groundingKey() | ||
| return (sequenceOf(name) + queries.asSequence()) | ||
| .map(String::groundingKey) | ||
| .any { queryName -> | ||
| candidateName == queryName || | ||
| candidateName.isCompatibleName(queryName) || | ||
| candidateName.isFuzzyNameMatch(queryName) || | ||
| (queryName.length >= MIN_NAME_COMPATIBILITY_KEY_LENGTH && candidateAddress.contains(queryName)) | ||
| } | ||
| } | ||
|
|
||
| private fun PlaceClue.hasNameEvidence(candidate: PlaceCandidate): Boolean { | ||
| val candidateName = candidate.name.groundingKey() | ||
| return candidateName.length >= MIN_GROUNDING_KEY_LENGTH && evidence.any { clueEvidence -> | ||
| clueEvidence.evidenceText.groundingKey().contains(candidateName) | ||
| } | ||
| } | ||
|
|
||
| private fun PlaceClue.hasCompatibleEvidence(candidate: PlaceCandidate): Boolean { | ||
| val candidateName = candidate.name.groundingKey() | ||
| val candidateAddress = candidate.address.groundingKey() | ||
| val candidateAddressKeys = PlaceAddressMatcher.addressKeys(candidate.address) | ||
| return evidence.asSequence().map(PlaceClueEvidence::evidenceText).any { evidenceText -> | ||
| val normalizedEvidence = evidenceText.groundingKey() | ||
| val containsName = candidateName.length >= MIN_GROUNDING_KEY_LENGTH && | ||
| normalizedEvidence.contains(candidateName) | ||
| val containsAddress = candidateAddress.length >= MIN_ADDRESS_GROUNDING_KEY_LENGTH && | ||
| normalizedEvidence.contains(candidateAddress) | ||
| containsName || containsAddress || | ||
| candidateAddressKeys.any { it in PlaceAddressMatcher.addressKeys(evidenceText) } | ||
| } | ||
| } | ||
|
|
||
| private fun String.isCompatibleName(other: String): Boolean = length >= MIN_NAME_COMPATIBILITY_KEY_LENGTH && | ||
| other.length >= MIN_NAME_COMPATIBILITY_KEY_LENGTH && | ||
| (contains(other) || other.contains(this)) | ||
|
|
||
| private fun String.isFuzzyNameMatch(other: String): Boolean = length >= MIN_FUZZY_NAME_LENGTH && | ||
| length == other.length && zip(other).count { (left, right) -> left != right } <= MAX_NAME_CHARACTER_DIFFERENCE | ||
|
|
||
| private fun String.groundingKey(): String = lowercase().filter(Char::isLetterOrDigit) | ||
|
|
||
| private const val MIN_GROUNDING_KEY_LENGTH = 2 | ||
| private const val MIN_NAME_COMPATIBILITY_KEY_LENGTH = 3 | ||
| private const val MIN_FUZZY_NAME_LENGTH = 4 | ||
| private const val MAX_NAME_CHARACTER_DIFFERENCE = 1 | ||
| private const val MIN_ADDRESS_GROUNDING_KEY_LENGTH = 6 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When
addressHintis a lot-number address such as서울 마포구 상수동 354-12 지하 1층 201호but the provider returns the same place's road address, both key sets are nonempty and disjoint, so the real candidate is rejected. This is especially likely because bothKakaoPlaceMapperandNaverPlaceMapperprefer the road address whenever one is available. Only hard-reject addresses expressed in comparable forms, or retain/provider-map both road and lot-number forms for equivalence checking.Useful? React with 👍 / 👎.