Pixels!

Basically we leaned the basics of keeping all the data for an image on the screen with a 1d array called pixels. Calling pixels is really weird because there are so many pixel arrays that can be present. You can have one for an image, for the canvas, for videos, pixels keeps a lot of numbers in long line and you can break it down and do all sorts of stuff with it.

What kind of stuff? Whatever you can figure out basically. Its hard to understand a (I am going to use the word image indescrimitibly the same way you would class a painting as "the image" or as painters as working with images.) Image as a 1d array. But we can break am image down atomically into a cartesian grid with 0,0 in the top corner and then it gets read straight across to the right until the page ends at whatever the distance of the rectangle is and we pick right back up on the at the left side of the page and repeat just like with english and it then goes on and on until the picture is known.

Each pixel is made up of 4 values. Red, Green, Blue, and Alpha. Each value can be between 0 and 255. So a pure red pixel would be (255,0,0,255) and a pure green pixel would be (0,255,0,255) and a pure blue pixel would be (0,0,255,255). The alpha value is the transparency of the pixel so 255 is fully opaque and 0 is fully transparent. So if you wanted a semi transparent red you could do (255,0,0,127) (vscode autocompleted this whole paragraph)

So to manipulate pixels you have to first load the pixels into the array with loadPixels() and then you can manipulate the pixels array directly. Each pixel takes up 4 spaces in the array so the first pixel is at index 0,1,2,3 and the second pixel is at index 4,5,6,7 and so on. So to get the red value of the first pixel you would do pixels[0] and to get the green value of the first pixel you would do pixels[1] and so on. Once you are done manipulating the pixels you have to call updatePixels() to update the canvas with the new pixel values.(vs code autocompleted that too lol)

Okay, anyways. There are some ways to get through it using nested for loops and a couple equations to isolate the index from x and y values. Or what else is really cool because then you can just count by four each time you go through. These were the things we were taught and so these were the foundation of the pixel array based image code my classmate Valarie and I wrote to manipulate our image for 60 seconds

I had been reading a lot of the p5js book and understanding a lot of the basic ideas of the program. I know some arduino c so the most difficult thing is wrapping my head back around classes, objects, and arrays (lol).

Coming up with ideas for the image was easy, understanding what's realistic as a way to code that image was very difficult. She wanted to do a transition from a picture to a framing, I wanted to talk about memory and how our mental images of people and events fade over time. We talked about how to do it with a picture of her cat and broke off after a couple hours of work on a zoom call some ideas on how to proceed. I focused on figuring out the fading from the outside in and she worked on getting the image transition.

The first way I was trying to approach the fade was to iterate around the loop as a spiral and in random instances create an object which would essentially be a circle which expanded out. My initial idea was to use nested loops to interate through the image and draw lines on each side using a counter to shrink the x and y of the for loop each time through. I was only able to replace the ouside of the canvas with a red line.

I spent talked to Ryan and Willaim about it and realized the approaches described for this type of walk is very difficult. Ryan changed how I viewed the problem by drawing the image with 0,0 at its center instead of the upper left corner. I could sort through the array one way using index and calculate the distance as I checked each pixel. William helped me with dist() and proposed to use lerping to change the color.

 
                        let photo;
                        let vangogh;
                        let time = 60 * 60;
                        let pixelOrder = [];
                        let index = 0;

                        function preload() {
                            photo = loadImage('photo.jpg');   
                            vangogh = loadImage('vangogh.jpg'); 
                        }

                        function setup() {
                            createCanvas(photo.width, photo.height);
                            photo.loadPixels();
                            vangogh.loadPixels();
                            pixelDensity(1);
                            for (let i = 0; i < photo.pixels.length / 4; i++) {
                                pixelOrder.push(i);
                            }
                            shuffle(pixelOrder, true);
                        }

                        function draw() {
                            loadPixels();
                            let step = int(pixelOrder.length / time);
                            for (let i = 0; i < step && index < pixelOrder.length; i++) {
                                let pix = pixelOrder[index];
                                let p = pix * 4;
                                pixels[p] = vangogh.pixels[p];
                                pixels[p + 1] = vangogh.pixels[p + 1];
                                pixels[p + 2] = vangogh.pixels[p + 2];
                                pixels[p + 3] = 255;
                                index++;
                            }
                            for (let i = index; i < pixelOrder.length; i++) {
                                let pix = pixelOrder[i];
                                let p = pix * 4;
                                pixels[p] = photo.pixels[p];
                                pixels[p + 1] = photo.pixels[p + 1];
                                pixels[p + 2] = photo.pixels[p + 2];
                                pixels[p + 3] = 255;
                            }
                            updatePixels();
                            }
                    

This was the sketch that Valarie came back to me with. She had replaced the image of her cat with the photograph and the painting. I was really struck by how haunting the photo of vangough is. I took some time to understand her code and realized that all I had to do was tweak the order of the index which was replacing pixels, which was created in the setup.

I saw that she was creating an array with all the pixel indexes and then going through that array and replacing the pixels one by one first drawing the new pixels and then drawing in the old pixels after that. I realized that I could just sort the array she created by each pixels distance from the center. I wrote up a version of the code tried to implament it and then asked chat to help me debug it. It threw me back a fixed algoritham which I spent time to understand. Each index of the array was ammended with a distance value and the array was then sorted by those values. From here I tweaked the values until it was smooth and borderless. I really wanted a slow zoom on the eye.

                    let photo;
                    let vangogh;
                    let time = 60 * 60;
                    let pixelOrder = [];
                    let index = 0;

                    function preload() {
                    photo = loadImage('photo.jpg');   
                    vangogh = loadImage('vangogh.jpg'); 
                    }

                    function setup() {
                    createCanvas(photo.width, photo.height);
                    photo.loadPixels();
                    vangogh.loadPixels();
                    pixelDensity(1);
                    
                    let xCent = width / 2;
                    let yCent = height / 2;
                    
                    let pixelsWithDist = [];
                    for (let i = 0; i < photo.pixels.length / 4; i++) {
                        let y = floor(i / width);
                        let x = i % width;
                        let d = dist(xCent, yCent, x, y);
                        pixelsWithDist.push({index: i, distance: d});
                    }
                    
                    let smoothAmount = 100;
                    pixelsWithDist.forEach(p => {
                        p.distance += random(-smoothAmount, smoothAmount);
                    });
                    
                    pixelsWithDist.sort((a, b) => b.distance - a.distance);
                    
                    for (let p of pixelsWithDist) {
                        pixelOrder.push(p.index);
                    }
                    }

                    function draw() {
                    loadPixels();
                    
                    let step = int(pixelOrder.length / time);
                    for (let i = 0; i < step && index < pixelOrder.length; i++) {
                        let pix = pixelOrder[index];
                        let p = pix * 4;
                        pixels[p] = vangogh.pixels[p];
                        pixels[p + 1] = vangogh.pixels[p + 1];
                        pixels[p + 2] = vangogh.pixels[p + 2];
                        pixels[p + 3] = 255;
                        index++;
                    }
                    
                    for (let i = index; i < pixelOrder.length; i++) {
                        let pix = pixelOrder[i];
                        let p = pix * 4;
                        pixels[p] = photo.pixels[p];
                        pixels[p + 1] = photo.pixels[p + 1];
                        pixels[p + 2] = photo.pixels[p + 2];
                        pixels[p + 3] = 255;
                    }
                    
                    updatePixels();
                    }
            

This was close to our final version that we presented! I reread the assignment and realized the image had to scale to the full screen size of the web page so Valerie added another line "createCanvas(windowWidth&height)" which scaled the picture to the full size of the window. Or so we thought. When it was time to present we were left with a hidious white mass to the side of the image which despite our teachers best last minute efforts remained unfixable for the viewing.

For our words to describe the transformation of the image we chose:

• Reality
• Self-expression
• Transformation
• Memory

It was great to present the project infront of others and see how they took it. I think memory was a difficult theme to convey. The pixeley nature of the transition was a little off topic as well. I think having the pixels fade in with lerping would have been more effective at communicating the more analog nature of the mind.

As I worked with the code, I commented out " //createCanvas(photo.width, photo.height);" and played with the window size to produce different results. I got really lucky a couple times and produced really striking repetition based images where vangoughs eye was centered along the middle of the picture and repeated over and over. If I were to extend this project, I would try to produce this result more reliably and dial it in. I think that this large horizontally focused picure is more cinematic and abstract.