New Content - Implemented Image path change due to new KoL hosting

lostcalpolydude

lostcalpolydude

Developer
Staff member
Jick mentioned on a recent radio show that KoL's hosting is moving. That means (at least on dev right now) a new URL path for images. In 16827 I made a few changes to make mafia usable again on dev.

It's possible these changes won't ever be visible outside of dev, in which case this will just be me changing things that are broken enough for someone affected by them to notice. There's also the possibility that the URLs will change, which would mean changing code all over the place.
 
I would guess that all images are accessible at both the old and new URL. It's just a question of which URL is used in the HTML.
 
I just happened to visit a Monster Manuel page and this flooded the gCLI:

Code: 
Manuel says that 'A.M.C. gremlin' (554) has unrecognized image ''
Manuel says that 'Abcrusher 4000™' (1487) has unrecognized image ''
Manuel says that 'Accountant-Barbarian' (1678) has unrecognized image ''
Manuel says that 'acid blob' (220) has unrecognized image ''
Manuel says that 'acoustic electric eel' (736) has unrecognized image ''
Manuel says that 'actual orcish frat boy' (956) has unrecognized image ''
Manuel says that 'Adventurer echo' (1595) has unrecognized image ''
Manuel says that 'aggressive grass snake' (1928) has unrecognized image ''
Manuel says that 'albino bat' (41) has unrecognized image ''
Manuel says that 'alert mariachi' (860) has unrecognized image ''
Manuel says that 'alielf' (1092) has unrecognized image ''
Manuel says that 'alien' (1724) has unrecognized image ''
Manuel says that 'alien hamsterpus' (1096) has unrecognized image

It's every image on every page I visited. I'm guessing that this change might be the reason why. But then again I'm only guessing. :)

I just realized that I was imprecise in my wording. The change I referred to is the one in KoL not KoLmafia and this should probably be a new thread.
 
Last edited:
All Monster Manuel images are now unrecognized

I just happened to visit a Monster Manuel page and this flooded the gCLI:

Code: 
Manuel says that 'A.M.C. gremlin' (554) has unrecognized image ''
Manuel says that 'Abcrusher 4000™' (1487) has unrecognized image ''
Manuel says that 'Accountant-Barbarian' (1678) has unrecognized image ''
Manuel says that 'acid blob' (220) has unrecognized image ''
Manuel says that 'acoustic electric eel' (736) has unrecognized image ''
Manuel says that 'actual orcish frat boy' (956) has unrecognized image ''
Manuel says that 'Adventurer echo' (1595) has unrecognized image ''
Manuel says that 'aggressive grass snake' (1928) has unrecognized image ''
Manuel says that 'albino bat' (41) has unrecognized image ''
Manuel says that 'alert mariachi' (860) has unrecognized image ''
Manuel says that 'alielf' (1092) has unrecognized image ''
Manuel says that 'alien' (1724) has unrecognized image ''
Manuel says that 'alien hamsterpus' (1096) has unrecognized image

It's every image on every page I visited. I'm guessing that the recent KoL change to image location might be the reason why. But then again I'm only guessing. :)
 
Threads merged because one is a special case of the other. Just... never mind. Not going to say anything else.
 
I wonder if the correct fix is to change various regexes to match images.kingdomofloathing instead of http://, and also to rewrite s3.amazonaws/images.kingdomofloathing URLs to also point to /images/.
 
Revision 16841 fixes the Monster Manuel regexp for monster image to accept any of the following:

src=http://images.kingdomofloathing.com/...
src=/images/...
src=https://s3.amazonaws.com/images.kingdomofloathing.com/...

Note that the aws image is fetched with https. I assume by rewriting, you are referring to methods like these:

FileUtilities.java:

Code: 
	private static final String localImageName( final String filename )
	{
		if ( filename == null || filename.equals( "" ) )
		{
			return null;
		}
		String localname = filename.substring( filename.indexOf( "/", "http://".length() ) + 1 );
		if ( localname.startsWith( "albums/" ) )
		{
			localname = localname.substring( 7 );
		}
		return localname;
	}
That takes a KoL url and extracts the path. Notice that it assumes http:

RelayRequest.java:

Code: 
	private static String localImagePath( final String filename )
	{
		return	filename.endsWith( "favicon.ico" ) ?
			"http://www.kingdomofloathing.com/favicon.ico" :
			filename.startsWith( "images" ) ?
			"http://images.kingdomofloathing.com" + filename.substring( 6 ) :
			filename;
	}
This converts one of our "rewritten" URLs to go back to the old image server.

RelayRequest.java:

Code: 
	public static final void overrideImages( final StringBuffer buffer )
	{
		if ( !Preferences.getBoolean( "relayOverridesImages" ) )
		{
			return;
		}

		for ( String filename : IMAGES )
		{
			String find = "http://images.kingdomofloathing.com/" + filename;
			String replace = "/images/" + filename;
			StringUtilities.globalStringReplace( buffer, find, replace );
		}
	}
Only detects old image server.

I see that lost added this to KoLmafia.java:

Code: 
	// Currently this setting is only meaningful when logging in to the
	// dev server.  More code changes will possibly be needed later
	public static boolean useAmazonImages = false;

I'm adding the following:

Code: 
	public String imageServerPath()
	{
		return  KoLmafia.useAmazonImages ?
			"https://s3.amazonaws.com/images.kingdomofloathing.com/" :
			"http://images.kingdomofloathing.com/";
	}
and using at as seems appropriate.

That won't solve the various regexps to work on either, but the change I made to MonsterManuelManager should work with either.
 
Veracity said:
Revision 16841 fixes the Monster Manuel regexp for monster image to accept any of the following:

src=http://images.kingdomofloathing.com/...
src=/images/...
src=https://s3.amazonaws.com/images.kingdomofloathing.com/...

Note that the aws image is fetched with https.
Click to expand...
In particular, I was thinking more along the lines of
Code: 
Pattern.compile( "<img src=[^>]*?(?:images.kingdomofloathing.com|/images)/(?:(adventureimages|otherimages)/(?:\\.\\./)?)?(.*?\\.gif).*?</td>" );
since it seems like the only difference to the URLs is that http:// is replaced by https://s3.amazonaws.com/

For what it's worth, the CNAME record for images.kingdomofloathing.com has recently been changed (I think it happened on the 1st, but I can't tell for sure) to point to the Amazon site.

And correct, my comment about "rewriting" was directed at overrideImages and associated methods -- thanks for digging those up; I'd probably have missed at least one of them.
 
Yeah, there is also RelayRequest.sendServerReply which actually rewrites.

OK, I am working on a patch for this, including fixing the regexps. I like yours better.
 
Well, revision 16843 should be able to handle either old or new image paths when parsing pages and will generate either the old or new in generated HTML, depending on whether lost's variable says to do so.

Given what you said about the CNAME record, that probably wan't necessary.

The Relay Agent also "rewrites" Amazon image URLs to go to our local image cache - "/images" - just as it did for the KoL image URLs.

For what I tested, "works for me". We'll see if others find issues...
 
And, revision 16845 fixes various aspects of that.

I tested with useAmazonImages set to true and to false. Both worked fine for me.

When true, we actually request images from the amazon image site, otherwise from the old image site - which now forwards to the new one, according to heeheehee.

We could probably change this line in GenericRequest:

Code: 
		KoLmafia.useAmazonImages = useDevProxyServer;
to

Code: 
		KoLmafia.useAmazonImages = true;

or simply remove it and initialize that variable to true. I can't test with the dev server, but on the regular server, we work fine (per my testing) with it set to true.
 
It's always true on the dev server currently, so that change couldn't break anything there.

I didn't really think my way of handling things was all that good, but it worked well enough temporarily to make things not completely broken on dev at the time. Removing that boolean seems like a good plan.
 
Well, what I meant was that apparently your terrarium comes back with amazon images on the dev server, but it does not on the normal server. I changed the pattern for parsing the terrarium to handle either, using heeheehee's method. I couldn't test that.

I changed everything appropriate to use KoLmafia.imageServerPath() for when we construct URLs, and I changed all the patterns that look for images to use heeheehee's method, so we should be resilient if KoL starts using the amazon path more or abandons it entirely.

If KoL continues to forward the old name to the new one, we could/should probably just continue to use the old one. That way, if KoL gives up on it or moves somewhere else - but still forwards - we'll be OK.
 
heeheehee said:
For what it's worth, the CNAME record for images.kingdomofloathing.com has recently been changed (I think it happened on the 1st, but I can't tell for sure) to point to the Amazon site.
Click to expand...

And, as graycat found out and posted in GD, this is a SSL error when used with https. The amazon servers are identifying themselves with ssl certificates for s3.amazonaws.com, not for images.kingdomofloathing.com. Therefore, browsers should reject requests to https://images.kingdomofloathing.com (or at least warn the user).
If mafia rewrites all image urls so that browsers always see https://s3.amazonaws.com/..., we will see fewer errors than vanilla browsing.
 
It is trivial to do that.

KoL says to use https://s3.amazonaws.com/images.kingdomofloathing.com/xxx or http://images.kingdomofloathing.com/xxx
KoLmafia tell the browser to fetch /images/xxx
The browser requests /images/xxx
KoLmafia asks for https://s3.amazonaws.com/images.kingdomofloathing.com/xxx or http://images.kingdomofloathing.com/xxx, depending on how it is configured - regardless of what KoL advertised.

I am currently running with useAmazonImages = true (note - not a preference. It's an internal variable) and I see us successfully fetching images using https://s3.amazonaws.com.

(With that turned off, we request http://images.hambo.com and that works too.)

I may just submit this. Lets try it out.
 
Revision 16847 does that.

When KoL says an image is on either the old image server or the amazon one, tell the browser it is in /images - KoLmafia's image cache.
When the browser asks for an image in /images, either serve the cached image or ask KoL for the image at either the old server (using http) or the amazon one (using https), cache it, and serve the image to the browser.
Use the amazon image server by default.

(Not configurable via preference, although I could do that.)

Works for me. Lets see how it works for you. ;)
 
You must log in or register to reply here.
Back
Top