Managing 4.2 billion pixels...

2023-04-26
,

In my previous post "The Big Bite" I wrote about working with 64K, meaning huge chunks of terrain in one go. I wrote that I decided to work using "floating" progression, start with a single tile. Enlarge the image to cover the next tile and work on it to match the first area, then save the first area as a separate file, enlarge again and repeat.  That was in February, now a few months and hundreds of hours later I know better. To effectively manage 4.2 billion pixels takes organization and persistence. To reshuffle pixels was OK to do once or twice, but to rely on in as a method was not a good idea with the same area being edited in to different places and lots of copping, copying mask and other hassles.

Back to the drawing board trying to come up with a better way of trying to edit 65,536 x 65,536 (which equals 4,294,967,296) pixels.  I decided to group four 8K areas into group that I name and keep in the same file all the time to make sure that each area is only edited in a single place. Below is an overview map of my Shield Land 1-64K area with the Groups in white and the XY of each 8K cell in yellow. I haven't named the top row yet, will get to them soon.

Next step was to develop a 16K (16,384 x 16,384 pixels) template, and come up with a way to include a bit of the surrounding areas to make sure that transition was seamless.  To get enough of information to make sure borders where seamless without too much overhead I decided to use 500 pixel wide border areas. I created a Photoshop file that had a 500 pixels extra in every direction. Guides both at the centers to accurately place the 8K textures and masks from World Machine, and guides both at the 16K marks and 500 pixels inside.

On the image below I have colored the surrounding area part yellow, and the 500 pixel margins from this 16K area pink. I can easily select, copy and paste this information between files. Take a look at the layers panel on the right and you can see that I placed all the layers I'm working on under Group 1, and given it a mask to make sure I don't edit outside of this 16K area. The surrounding areas pated in are above this group.

This is technical, I know, but to to set up your workflow is crucial when you work with lots of huge files and more data than you can keep track of. Naming conventions and storage strategies (including backup plans) are vital, sometimes it takes weeks, months or even years between the times you access a file. It would be hard to near impossible to find and know hot it is set up without a plan and organization. Writing blog posts like this also helps me remember how I was thinking, and then rethinking things when things needed improvement. That is a whole other topic, constant improvement.

Each 64K area are 4.2 billion pixels and the Flanaess alone with require a hundred or so areas...

You can download my Photoshop 16K template here (PSB format  17GB) https://www.dropbox.com/s/alxkb5069gwi899/Shield%20Lands%201%20-%2064K%20-%20South%20Keep%20-%201.psb?dl=0

Or you can opt for a scaled down 25% size version (PSD format 1.29GB) here: https://www.dropbox.com/s/8ejlwxzz98wplbi/Shield%20Lands%201%20-%2064K%20-%20South%20Keep%20-%201%4025%25.psd?dl=0

I'll write a detailed guide to how the file is structured soon.