BeachComber - fast and efficient beach combing

(9063,3,7) is a rare?

Code: 
> BeachComberUtilities combed verbose

Loading rare tile data...
Loading verified rare tile data...
Loading uncommon tile data...
Loading common tile data...
Loading beach head data...
Loading sand castle data...
Loading combed tile data...
Done loading tile data
There is 1 unknown combed tile on 1 beach.
    (9063,3,7)

That is, literally, the only tile I have only seen as "combed" which is not in combo's raretiles.json.

By the way - once I have seen all "combed" tiles, I will update my data files to make columns run from 1-10, rather than 0-9.
I sort of understand why combo did that - the _beachLayout property lists rows from 1-10, but then gives a 10-character string for each row - and the character index in the string runs from 0-9 - but that's a computer-readability thing, not a sensible human-readable notation.
 
Funny. So 1001 rare tiles.
We could have a false rare, but frankly it should've been found by now and I'm assuming you've have mentioned it if you found it.

As Rinn said in discord, 1001 is similar enough to 11
Funny enough of a number, 1002 and 999 is just weird.

I have 198 rares to verify, no other tiles remaining.

Code: 
1/24/2024, 11:50:36 AM  1706050236      Sand: 949,500
1/24/2024, 11:50:36 AM  1706050236      Uncommon Sparkle: 48,976
1/24/2024, 11:50:36 AM  1706050236      Rare Sparkle: 803
1/24/2024, 11:50:36 AM  1706050236      Castle: 512
1/24/2024, 11:50:36 AM  1706050236      Unknown tiles: 198
1/24/2024, 11:50:36 AM  1706050236      Head: 11
 
I added that "combed" tile to tiles.rare.json (which now has 1001 entries) and removed it from tiles.combed.json (which now has 0 entries).
I therefore removed detection of new combed tiles; the only remaining ones are simply unverified rares.
I changed "spade all" to go after unverified rares rather than "combed" - and stop after it has looked at all of them.
I also removed "combed" and "uncommon" from BeachComberUtilities, since they are no longer useful.

Code: 
Iterating over 10,000 beaches, 10 rows, 10 columns

Total rares = 1001
(Unverified rares = 377)
Total uncommons = 48976
Total commons = 949500
Total beach heads = 11
Total sand castles = 512
Total: 1000000

You have 198 unverified rares and I have 377 of them. Other than that, aside from terminology, we are in agreement.

I guess I am gradually removing "spading" features. Once I have personally seen all 1,000,000 tiles, MY spading will be done.
I'll adjust the data format to be more human (rather than machine) readable, and tiles.{rare,uncommon,common,head,castle}.json will be the ground truth for every tile on the beach.

And BeachComber will simply be functionally equivalent to a more efficient version of combo. :)
 
Revision 37 adds another 51 verified rares. It also publishes tiles.combed.json, which contains the remaining 332 unverified rares.
BeachComber spade all will visit only beaches with unverified rares that are not washed by the tides.

Next revision:

- I'll be simplifying the format of the various tiles.xxx.json files to be much more compact and, presumably, much faster to load.
- I'll also be generating a tiles.txt data file which has a million lines keyed 1-1000000 and a single string - rare, uncommon, common, head, castle, combed. "combed" is assumed to be "rare", but is not yet verified (by me). That will readable by file_to_map().

Code: 
Total rares = 1001
(Unverified rares = 332)
Total uncommons = 48976
Total commons = 949500
Total beach heads = 11
Total sand castles = 512
Total combed = 332
Total: 1000000

rowrare(combed#)uncommoncommonheadcastle#
101837776499294825000
91726666480895017300
81334747489794962800
7117454548459494509384
6994242491094991000
5602828492895012000
47919194931947940196156
36744488495049000
25422484195105000
137224940948000223178

Accumulating tidal data for 10 rows

rowrare(combed#)uncommoncommonheadcastle#
101837776499294825000
93551431429800189842300
8488190189146972848041100
760523523319542379749119384
670427727424452474740119384
576430530229380569752119384
48433243193431166454611289164
39103283233919575959511289164
29643303254403685470011289164
110013323274897694950011512180
 

Irrat's Beach Retreat

libraryaddict.github.io libraryaddict.github.io

Here we go. It could be improved and definitely could have the source tiles and whatnot included and compiled into said packed version..
But there's only so much effort its worth.
I'm currently on like, 52 rares left to verify. But I just marked every unverified tile as a rare for the built file.
 
I've been trying to "verify" all 1001 "rare" tiles, and have been stuck at 1000 for about 6 months.

I just learned a hitherto hoarded mechanic: there is always one rainbow pearl "rare" tile - but it takes two years to regenerate.

Which is to say, I've been trying to farm the next rainbow pearl.
I am quite certain that other people have already discovered this tile and are doing the same.

In any case, I believe the "spading" aspect of this script is complete.

- I removed the "spade" command
- I removed the "prune" and "merge" commands.
- ```BeachComber data``` no longer reports on "new" (rare, uncommon,common,castle) tiles or "verified rare" tiles.
- ```BeachComberUtilities complete``` is a bit more concise
- I cleared the "combed" data file; (rare, uncommmon, common, beach_heads, castle) data files contain all 1,000,000 tiles.

It strikes me that once the next rainbow pearl is found, the same handful of people will still want to spade out where the NEXT one will appear. So I added "pearl" mode.

```BeachComber pearl all``` (or ```BeachComber pearl N```) will spend the specified number of turns visiting only beaches that have "previously seen as combed" rare tiles. Where does it get that list? Well, if the "combed" data file is empty - as it is now - the first time you use ```BeachComber pearl```, it will initialize it with the complete set of rare tiles, and then act just like ```BeachComber rare``` - go only to beaches with rares - but will filter out tiles you have already found a rare in.

Which is to say, you will gradually visit all the rares - and harvest them - and eventually, just as I did, you'll get down to a single tile which stubbornly remains combed.

And that's your huckleberry.

Code: 
> set VBC.ParseCommons=true

VBC.ParseCommons => true

> BeachComber data

Loading tile data...
Loading rare tile data...
Loading uncommon tile data...
Loading common tile data...
Loading beach head data...
Loading sand castle data...
Loading combed tile data...
Done loading tile data

Rare tiles: 1001
Uncommon tiles: 48976
Common tiles: 949500
Beach Heads: 11
Sand Castles: 512
combed tiles: 0

Beaches with rare tiles: 951
Beaches with uncommon tiles: 9946
Beaches with common tiles: 10000
Beaches with beach heads: 11
Beaches with sand castles: 180
Beaches with combed tiles: 0

and

Code: 
> BeachComberUtilities complete

Loading rare tile data...
Loading uncommon tile data...
Loading common tile data...
Loading beach head data...
Loading sand castle data...
Loading combed tile data...
Done loading tile data

Iterating over 10,000 beaches, 10 rows, 10 columns

Total rares = 1001
Total uncommons = 48976
Total commons = 949500
Total beach heads = 11
Total sand castles = 512
Total combed = 0
Total: 1000000



rowrare(combed#)uncommoncommonheadcastle#
1018300499294825000
917200480895017300
813300489794962800
71170048459494509384
69900491094991000
56000492895012000
479004931947940196156
36700488495049000
25400484195105000
137004940948000223178

Code: 
Accumulating tidal data for 10 rows



rowrare(combed#)uncommoncommonheadcastle#
1018300499294825000
9355009800189842300
848800146972848041100
76050019542379749119384
67040024452474740119384
57640029380569752119384
4843003431166454611289164
3910003919575959511289164
2964004403685470011289164
11001004897694950011512180


BeachComber.ash r43
 
Veracity said:
I've been trying to "verify" all 1001 "rare" tiles, and have been stuck at 1000 for about 6 months.

I just learned a hitherto hoarded mechanic: there is always one rainbow pearl "rare" tile - but it takes two years to regenerate.

Which is to say, I've been trying to farm the next rainbow pearl.
I am quite certain that other people have already discovered this tile and are doing the same.
Click to expand...
{ "minute": 9063, "row": 3, "column": 6 },
That's the one you think is a rainbow yeah? I remember seeing that discussion in kol chat, and I believe that's the only rare I added without confirming, because I couldn't confirm it. Obviously, none of the tiles I confirmed gave a pearl.
It also makes sense that rares are 1000, and the 1001th is not actually a rare.
Could be worth removing from combo, hoarded spot & super long cooldown.
 
Irrat said:
{ "minute": 9063, "row": 3, "column": 6 },
That's the one you think is a rainbow yeah? I remember seeing that discussion in kol chat, and I believe that's the only rare I added without confirming, because I couldn't confirm it. Obviously, none of the tiles I confirmed gave a pearl.
Click to expand...

No. For some reason, in the data files, "column" is numbered 0 - 9 but KoLmafia logs them as columns 1 - 10.

Code: 
> Combing the square at coordinates (9063,3,7) which contains rough sand with a twinkle
Combing square 3,7 (9063 minutes down the beach)
You acquire an item: piece of driftwood
> (9063,3,7) is a 'rare' tile.

The single tile I have been unable to verify is

Code: 
  { "minute": 9966, "row": 8, "column": 4 }

Which is to say, (9966,8, 5)

Irrat said:
It also makes sense that rares are 1000, and the 1001th is not actually a rare.
Could be worth removing from combo, hoarded spot & super long cooldown.
Click to expand...

I believe the mechanism is this:

- There are 1001 rare tiles.
- One is designated as the "next rainbow pearl" and is marked combed with a long cooldown.
- When it becomes uncombed and somebody combs the pearl, a different combed rare becomes the "next rainbow pearl".

If this script finds a pearl, it will log it - as a rare - and clear the combed tiles data file.
That will reload that data file with all rare tiles, next time you run in "pearl" mode, since you need to re-spade the "next rainbow pearl" rare.

I don't have an opinion on combo. One out 1001 tiles it takes you to will be effectively dead - but which tile that is will change, if somebody finds a rainbow pearl. Which COULD be a combo user, but is more likely to be a hoarder.
 
Veracity said:
The single tile I have been unable to verify is

Code: 
  { "minute": 9966, "row": 8, "column": 4 }

Which is to say, (9966,8, 5)
Click to expand...
This was the tile that I identified back in September of last year, and I indeed found a rainbow pearl in it just under 3 months ago. I suspect the next one must necessarily be in a different location because...

Veracity said:
I believe the mechanism is this:

- There are 1001 rare tiles.
- One is designated as the "next rainbow pearl" and is marked combed with a long cooldown.
- When it becomes uncombed and somebody combs the pearl, a different combed rare becomes the "next rainbow pearl".
Click to expand...
My running guess is that it works roughly as follows
  1. You find a rare in spot (X,Y,Z)
  2. That category of rare has some cooldown rolled with bounds based on the category of rare and is placed in (A,B,C)
  3. The next (A,B,C) is set to the current (X,Y,Z)
So yes, my hypothesis agrees with yours.

I guessed the bounds for rainbow pearls average to two years, since we know one was found early on (in 2019) and two more were found by end of 2023 (as far as we can see from Rupa's display case, who has obviously worked this out a long time ago and kept it to himself). I found mine in 2025, so I estimate some time in early 2027 it will become available but I have no idea what the lower bound is and I'm not sure it will ever be possible to tell.
 
Fascinating. If you found a rainbow pearl 3 months ago (while I, too was actively going there; good to be in the UK, eh? 😉) - and the next pearl is not there any more, I guess I’ve been normally unlucky at not finding a different rare there.

Time to restart pearl spading to find NEXT tile.
 
More spading.

1) I added six new files containing lists (allowing duplicates) on coordinates where we have found 6 categories of rare tiles:

tiles.rare.driftwood.json
tiles.rare.pirate.json
tiles.rare.message.json
tiles.rare.whale.json
tiles.rare.meteorite.json
tiles.rare.pearl.json

2) I updated BeachComberFix (my utility which parses session logs and updates BeachComber data files)

Code: 
> BeachComberFix rare save

Processing logs for CHAR1
Processing logs for CHAR2
Processing logs for CHAR3
Processing logs for CHAR4
Processing logs for CHAR5
Processing logs for CHAR6

Total rares collected: 1370
beached whale: 10
cursed pirate hoard: 64
message in a bottle: 35
meteorite fragment: 2
piece of driftwood: 1259

Those were the characters that did all of my initial spading:

1) iterating across all 10,000 beaches and identifying commons, castles, heads - and visiting unknown twinkles.
2) By and by, visit only beaches with unidentified squares.
3) And eventually, one the unidentified squares were doable by a single character, I assigned one character to that task

3) I updated BeachComber to log that data:

Code: 
> BeachComber rarities

Driftwood: 1259 (931 tiles, 886 beaches)
Pirate hoards: 64 (62 tiles, 62 beaches)
Message bottles: 35 (35 tiles, 35 beaches)
Whales: 10 (10 tiles, 10 beaches)
Meteorite fragments: 2 (2 tiles, 2 beaches)
Rainbow pearls: 0 (0 tiles, 0 beaches)

4) BeachComber will now update those files, as appropriate, when it finds a rare.

5) And some spading:

Code: 
> BeachComberUtilities rarities


TYPEdriftwoodpiratemessagewhalemeteoritepearlTOTAL
driftwood890 (840)28 (30)11 (14)1 (1)1 (1)0 (0)931 (886)
pirate28 (30)32 (30)2 (2)0 (0)0 (0)0 (0)62 (62)
message11 (14)2 (2)22 (19)0 (0)0 (0)0 (0)35 (35)
whale1 (1)0 (0)0 (0)9 (9)0 (0)0 (0)10 (10)
meteorite1 (1)0 (0)0 (0)0 (0)1 (1)0 (0)2 (2)
pearl0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)

Note that the "total" agrees with the counts in the files.
If I change the table to not have duplicate cells:

TYPEdriftwoodpiratemessagewhalemeteoritepearlTOTAL
driftwood890 (840)28 (30)11 (14)1 (1)1 (1)0 (0)931 (886)
pirate0 (0)32 (30)2 (2)0 (0)0 (0)0 (0)34 (32)
message0 (0)0 (0)22 (19)0 (0)0 (0)0 (0)22 (19)
whale0 (0)0 (0)0 (0)9 (9)0 (0)0 (0)9 (9)
meteorite0 (0)0 (0)0 (0)0 (0)1 (1)0 (0)1 (1)
pearl0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)

The TOTAL column counts unique coordinates. Note that it adds up to 997.
I could have sworn I have visited 1000 of the 1001 rares.
Huh.

Conclusions:

1) That might actually be listed in order of increasing rarity
2) Notice that all rarities have had the same tile combed at least once and a piece of driftwood was found.
3) There are a fair number of rarer-than-driftwood tiles where I have not also combed driftwood - but that is almost certainly an artifact of my testing, If I had no unidentified tiles in a particular beach, while I was spading, I never returned to it. There are quite a few tiles with multiple rares:

Code: 
> BeachComber data

Loading tile data...
Loading rare tile data...
Loading uncommon tile data...
Loading beach head data...
Loading sand castle data...
Loading combed tile data...
Done loading tile data

Rare tiles: 1001
Uncommon tiles: 48976
Common tiles: 0
Beach Heads: 11
Sand Castles: 512
combed tiles: 997

Beaches with rare tiles: 951
Beaches with uncommon tiles: 9946
Beaches with common tiles: 0
Beaches with beach heads: 11
Beaches with sand castles: 180
Beaches with combed tiles: 947

50 of them, in particular.

4) Conclusion: Any rare tile can have any rarity.

I found 64 pirate hoards in 62 different tiles - but doesn't mean that two tiles are ONLY pirate hoards.

Thoughts:

Is there a limit - or fixed number - on how many of a given rarity can be pending?
- Yes, we think, for rainbow pearls: exactly one.
- How about meteorite fragments and whales?

Does the cooldown differ for each catagory of tile?
- Could make another data file sorted by log-date, but my data is mostly not-repeating files I combed, so it wouldn't be that useful.

Perhaps I should dedicate another character to doing "rare all", rather than "pearl all".
And I should probably adjust BeachComberFix so it can parse beach logs that are not generated specifically by this script.
Even combo - or custom script - users could auto-scrape their rares, then.
 
FWIW, I made BeachComberFix's log parsing capable of handling any COMB transaction after Dec 28, 2019, which is when I modernized KoLmafia's tile combing logging. This means I can now parse old BeachComber, combo, and modern BeachComber.

I can deduce coordinates and rarity of the square from just the following:

Code: 
Combing square 5,1 (8322 minutes down the beach)
You acquire an item: taco shell
Code: 
Combing square 2,3 (355 minutes down the beach)
You gain 11,316,935 Meat
Code: 
Combing square 9,5 (5540 minutes down the beach)
You acquire an item: cursed pirate cutlass

The only "tricky" thing is message in a bottle, which new BeachComber logs like this:
Code: 
Combing square 8,2 (1163 minutes down the beach)
> You found a message in a bottle!

except for a period when it logged that message somewhere else and I don't see a good way to pick it up.
I only miss 4 message bottles if I skip those.

I reran BeachComberFix with start date set to Dec 29, 2019 and picked up a whole lot of combing with my old BeachComber and with combo.

Code: 
> BeachComberUtilities rarities verbose

Driftwood: 1333 (936 tiles, 890 beaches)
Pirate hoards: 66 (64 tiles, 64 beaches)
Message bottles: 31 (31 tiles, 31 beaches)
Whales: 13 (13 tiles, 13 beaches)
Meteorite fragments: 3 (3 tiles, 3 beaches)
Rainbow pearls: 0 (0 tiles, 0 beaches)


TYPEdriftwoodpiratemessagewhalemeteoritepearlTOTAL
driftwood889 (839)30 (32)10 (12)5 (5)2 (2)0 (0)936 (890)
pirate0 (0)31 (29)2 (2)1 (1)0 (0)0 (0)34 (32)
message0 (0)0 (0)19 (17)0 (0)0 (0)0 (0)19 (17)
whale0 (0)0 (0)0 (0)7 (7)0 (0)0 (0)7 (7)
meteorite0 (0)0 (0)0 (0)0 (0)1 (1)0 (0)1 (1)
pearl0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)

This reinforces my observation that any rarity can appear on any rare tile square; look at whales and meteorite fragments.
 
1) KoLmafia r28508 made KoLmafia log messages in a bottle and I removed BeachComber logging of such.
I found such a message today and it worked like a charm.
2) BeachComberFix rare save will now parse all session logs back to Dec 29, 2019 and extract all rare tiles you found and save them in the six rare tile category files I mentioned above. BeachComber, combo, custom scripts, "beach" command, and even the Relay Browser, all work just fine; it only looks at KoLmafia-generated log entries.
3) Having done so:
Code: 
> BeachComber rarities

Driftwood: 1342 (937 tiles, 891 beaches)
Pirate hoards: 67 (65 tiles, 65 beaches)
Message bottles: 32 (32 tiles, 32 beaches)
Whales: 13 (13 tiles, 13 beaches)
Meteorite fragments: 3 (3 tiles, 3 beaches)
Rainbow pearls: 0 (0 tiles, 0 beaches)

Here is the PR, with more verbose description of what changed.

github.com

rare tile types by Veracity0 · Pull Request #44 · Veracity0/beach

Categorize rare tile "types" static rare_type NOT_RARE = "not rare"; static rare_type RARE_DRIFTWOOD = "piece of driftwood"; static rare_type RARE_PIRATE = "curs...
github.com github.com

I think this round of updates is done. Back to simply using free combs to look for rares as part of breakfast and daily pearl diving for one of my characters who has nothing better to do.
 
Looks like the regex in create_combing_matcher fails if someone has preference logging turned on since that could add one or more lines between the "Combing square" line and the results line in the logs, for example this isn't matched:

Code: 
Combing square 2,3 (7392 minutes down the beach)
Preference _freeBeachWalksUsed changed from 5 to 6
You acquire an item: piece of driftwood

Other preferences I noticed that could be there were _beachLayout and _beachHeadsUsed but there may be others. Easiest fix is probably just to sanitize the input string to remove "Preference .*?\n"before the regex search.
 
Last edited:
Thanks! I think sanitizing each log file before processing it would be the simplest fix, as you suggest.
I'll do that, test, and submit before I run my turns tomorrow morning.

I'll probably remove the vestiges of the previous use for this file: processing all the logs and determining which times I'd visited of each type, for the purpose of initially populating my data files of rares, uncommons, commons, and castles. Since the spading is complete for all of those, it's really not useful.

I've considered making a new option to log at all the log files and tallying up all the loot, and saving it per player and in-total.
I've also considered changing the format of the file I already save stuff like that in - in which case that would actually be useful.
 
You must log in or register to reply here.
Back
Top