Fried Kielbasa

Obviously, each of these works influenced me at most by ~0.5% on an imaginary scale. That still is a lot in my opinion for just reading a book or watching a movie. I’m going to try to optimize for works with a maximum influence per obviousness ratio. For example, Star Wars obviously affected me a lot but it affected everyone else too, so it isn’t a particularly interesting work to list here.

Books

• Arthur Clarke’s short stories – Scifi in it’s purest form – Blew my mind as a kid. Singlehandedly inspired me to learn how technology works and got me excited for the future.
• The Lion of Comarre – One specific Arthur Clarke short story – Made me consider what I want the future to look like. Got me excited about robotics and how that would reshape society.
• Matterhorn – Vietnam war novel – There but for the grace of God go I. It was an experience reading this.
• How To Be A Victorian – A historical book detailing how Victorians actually lived – Opened my eyes to how people in the past weren’t any different from us today, they just operated under different incentives and constraints.
• The Short Happy Life of Francis Macomber – Short story by Hemingway about courage – I read this at a pivotal time in my teenage years when I was taking on a more risk-taking attitude.

Movies

• The Secret Life of Walter Mitty – Man goes from daydreamer to global adventurer – Seriously influenced me to become more adventurous as a person.
• Tampopo – A Japanese ramen western – It inspired me to take food more seriously. It has an outsized effect on me anyways, I might as well give it the respect it calls for.
• The Family Man – I can’t figure out what the moral of this story is. Every few years I change my mind on which path I would’ve chosen for myself. I think when I watched this as a kid it made me consider what role working plays in life.

TV Shows

• Mad Men – A show about an ad agency in the 60’s – The only show where I felt like I KNEW the characters. I haven’t ever rewatched it because my memory of watching it the first time is so mindblowing.

Games

• The Total War series – Historical RTS games set in Feudal Japan/Fall of Rome etc – Inspired my love of history.

Implementing the Harris Corner Detector

As an introduction to OpenCV and using it with modern C++, I decided to code a Harris corner detector. I’d previously only used MexOpenCV so this was new to me. I’m 100% certain that this could’ve been done more efficiently but I think that I should prioritize moving on to new material rather than perfecting this. Quality vs quantity. This was also my first introduction to makefiles and gdb, but I’ll include that elsewhere.

My main problem when coding this was that I kept mixing up types for cv::Mat. This website was so incredibly helpful for me. I can’t even begin to explain how many errors I had where it was simply because I was mixing up Mat types. I’m not certain why the compiler doesn’t throw an error when this happens, but I might switch to a different one. I also found the at function strange in OpenCV, as in image.at(i, j). Why can’t the be type deduced?

These are the general steps of the Harris Corner Detector

1. Take the grayscale of an image
2. Apply the Sobel operator to find the gradient values at each pixel
3. Compute the gradient covariance matrix elements
4. Apply gaussian blur to the covariance matrix elements
5. Calculate Harris score
6. Threshold and apply non maximum suppression

I’ve seen sources apply Gaussian blur to the image after step 1 as well.

I wanted to learn this from a sort of first principles approach, so I started with coding a Sobel operator. This is a method of finding the x and y gradients at every pixel of an image. Functionally, this detects edges in an image, which is useful because corners are the intersection of edges. How it works is you take a specific kernel (matrix) and multiply element wise with a 3×3 patch of the image.

Sobel operator kernel

My implementation iterates over each pixel in the image. I hardcoded the kernel, as opposed to creating a Mat, simply because it seemed simpler. I created two temp Mats of int type to store the output. This was necessary because the output of multiplication like this would’ve overflowed uchar. At the end I cast and scale the temporary Mat data to be back to uchar for consistency. You can see the results of running on a chessboard image below.

The original image

X Gradient

Y Gradient

I then needed to calculate the gradient covariance matrix, which is this

If you remember that the Sobel operator calculates the x and y gradients at each pixel, this is just going to be iterating over each pixel in the gx and gy Mats and multiplying as required. I used the mul function, which does element wise multiplication.

This was one tricky part for me. I had gotten to the part where I had to calculate the Harris score, which is determined by the equation below.

I had a problem though. Wouldn’t the determinant of M always be 0? What needs to happen before I do the Harris score calculation is I need to apply a window function to M.

For this I chose a Gaussian blur with radius of 2. Gaussian blur is simply applying a 3×3 kernel to a 3×3 image patch again, similar to the Sobel operator. At the end you multiply by the inverse of the sum of the matrix elements, to compute an average.

Gaussian blur kernel

I tried to do something a little strange here. Rather than hardcoding the values of the Gaussian kernel, I created a loop to fill the values in for me. If I had to do this next time, I would probably hardcode the values in a Mat, then use the mul function. I don’t think I would actually precalculate the values from the Gaussian function, since it’s not too difficult to just hardcode.

Blurred XX Gradient

Blurred XY Gradient

Blurred YY Gradient

Now we can calculate Harris score. I use k constant of 0.04 since that’s what was recommended here. I then create a float Mat called window and fill it with the elements of the blurred gradient covariance matrix. Again, its type float to avoid overflow when calculating trace and determinant. I then threshold based on an empirically determined value (I chose it based on when I was getting a reasonable number of corners). It’s important to note that both very negative and very positive values are what you’re looking for. I then put the absolute value of the score into a Mat. I was considering using a std::vector<cv::Point>, but I wanted the score, as well as the coordinates.

I then ran a quick and not very good form of non maximum suppression. The idea is to find the best corner in an area, and then suppress (ignore) all the others. I iterated over the corners Mat with a 40×40 window and placed the point with the highest score into a std::vector<cv::Point>. I have a gut feeling this could’ve been done much more efficiently, without needing a Mat with all the corners but I’m certain I’ll have the opportunity to reimplement this at some point in the future. The major issue is at the intersection of the 40×40 windows. The image below shows what happens.

I then put all of my corners onto the color image for easy viewing. I seem to have issues with not detecting the corners perfectly on center, as well as the previously mentioned non max suppression problem. This OpenCV tutorial details how sub pixel accuracy can be achieved.

I’ve added my full code below.

#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <iostream>
#include <vector>
#include <math.h>

std::vector<cv::Point> nonMaxSuppression(cv::Mat corners){
  cv::Mat window = cv::Mat(40, 40, CV_64F);
  std::vector<cv::Point> points;
  double maxval;
  cv::Point point;
  for(int i = 0; i < corners.cols - 40; i += 40){
    for(int j = 0; j < corners.rows - 40; j += 40){
      window = cv::Mat(corners, cv::Rect(i, j, 40, 40));
      minMaxLoc(window, NULL, &maxval, NULL, &point);
      if(maxval != 0)
        points.push_back(cv::Point(point.x + i, point.y + j));
    }
  }
  return points;
}

void cornerDetect(cv::Mat ixx, cv::Mat ixy, cv::Mat iyy, cv::Mat corners){ //change corners to float??
  const int k = 0.04;
  cv::Mat window = cv::Mat(2, 2, CV_32F);
  for(int i = 0; i < ixx.cols; ++i){
    for(int j = 0; j < ixx.rows; ++j){
      float Ixx = (float) ixx.at<uchar>(cv::Point(i, j));
      float Ixy = (float) ixy.at<uchar>(cv::Point(i, j));
      float Iyy = (float) iyy.at<uchar>(cv::Point(i, j));
      window.at<float>(0, 0) = Ixx;
      window.at<float>(0, 1) = Ixy;
      window.at<float>(1, 0) = Ixy;
      window.at<float>(1, 1) = Iyy;
      double score = cv::determinant(window) - k * cv::trace(window)[0] * cv::trace(window)[0];
      if(score > 20000 || score < -20000)
        corners.at<double>(cv::Point(i, j)) = std::abs(score);
    }
  }
}

void gaussBlur(cv::Mat image, cv::Mat blurredimage){
  for(int i = 3; i < image.cols - 3; ++i){
    for(int j = 3; j < image.rows - 3; ++j){
      int val = 0;
      for(int h = - 1; h < 2; ++h){
        for(int k = - 1; k < 2; ++k){
          int vertical = (h == 0) ? 2 : 1;
          int horizontal = (k == 0) ? 2 : 1;
          val += (int)image.at<uchar>(cv::Point(h + i, k + j)) * vertical * horizontal;
        }
      }
      blurredimage.at<uchar>(cv::Point(i, j)) = val / 16; 
    }
  }
}

void sobel(cv::Mat image, cv::Mat gx, cv::Mat gy){
  cv::Mat temp1 = cv::Mat(image.cols, image.rows, CV_32S);
  cv::Mat temp2 = cv::Mat(image.cols, image.rows, CV_32S);
  for(int i = 3; i < image.cols - 3; ++i){
    for(int j = 3; j < image.rows - 3; ++j){
       temp1.at<int>(i, j) = -1 * image.at<uchar>(i - 1, j - 1) - 2 * image.at<uchar>(i - 1, j) - image.at<uchar>(i - 1, j + 1) + image.at<uchar>(i + 1, j + 1) + 2 * image.at<uchar>(i + 1, j) + image.at<uchar>(i + 1, j - 1);
       temp2.at<int>(i, j) = image.at<uchar>(i - 1, j - 1) + 2 * image.at<uchar>(i, j - 1) + image.at<uchar>(i + 1, j - 1) - image.at<uchar>(i + 1, j + 1) - 2 * image.at<uchar>(i, j + 1) - image.at<uchar>(i - 1, j + 1);
    }
  }
 cv::convertScaleAbs(temp1, gx);
 cv::convertScaleAbs(temp2, gy);
}

int main(){
  cv::Mat image, colorimage;
  image = cv::imread("image.png", 0);
  colorimage = cv::imread("image.png", 1);
  cv::namedWindow("display window", cv::WINDOW_NORMAL);
  cv::Mat gx(image.cols, image.rows, CV_8UC1);
  cv::Mat gy(image.cols, image.rows, CV_8UC1);
  sobel(image, gx, gy);
  cv::Mat gxx = cv::Mat(image.cols, image.rows, CV_8UC1);
  cv::Mat gyy = cv::Mat(image.cols, image.rows, CV_8UC1);
  cv::Mat gxy = cv::Mat(image.cols, image.rows, CV_8UC1);
  gxx = gx.mul(gx);
  gyy = gy.mul(gy);
  gxy = gx.mul(gy);
  cv::Mat blurredgxx = cv::Mat(image.cols, image.rows, CV_8UC1);
  cv::Mat blurredgxy = cv::Mat(image.cols, image.rows, CV_8UC1); 
  cv::Mat blurredgyy = cv::Mat(image.cols, image.rows, CV_8UC1);
  gaussBlur(gxx, blurredgxx);
  gaussBlur(gxy, blurredgxy); 
  gaussBlur(gyy, blurredgyy);
  cv::Mat corners = cv::Mat(image.cols, image.rows, CV_64F);
  cornerDetect(blurredgxx, blurredgxy, blurredgyy, corners);
  std::vector<cv::Point> points = nonMaxSuppression(corners);
  for(auto i : points)
    cv::circle(colorimage, i, 7, (0,0,255));
  cv::imshow("display window", colorimage);
  cv::waitKey(0);
  return 0;
}

I took ARH 206 at Stony Brook back in Fall 2019 with Prof. David Mather and it was the most impactful class I took in college. I specifically took the class because I thought modern art was bullshit. I’m very happy to say that the class completely changed my view of modern art – specifically seeing the interplay between various artistic movements, how they affected and were influenced by popular culture. I’m writing this from the perspective of an engineer and layperson who appreciates art, so Tweet at me if you disagree with anything. Note – I lump contemporary art in with modern art. My intention with this piece is to convince people that the weird shit in art galleries is pretty cool, not to debate the finer points.

I was always confused by modern art. I didn’t understand the point of it. Older art showed mastery of technical skills, as well as beauty that everybody could appreciate. When you look at the work of the Old Masters, you can clearly see the years of effort and work it took to create such incredibly intricate work. Modern art was always a little weird and pretentious, besides it looked like a 5 year old could make it. I could display a canvas painted white with a black square in the middle myself and nobody would give a shit, so is modern art just marketing? If you take a step back, you notice what I value in art — beauty, technical skill, effort. Now this brings up the question of what exactly is art.

Between you and me, I don’t think anyone knows exactly what art is and anyone who tells you otherwise is full of shit. Let’s rip off Wikipedia –

Art is a diverse range of human activities in creating visual, auditory or performing artifacts (artworks), expressing the author’s imaginative, conceptual ideas, or technical skill, intended to be appreciated for their beauty or emotional power.

This sounds pretty legit to me since it’s a wide open definition and it definitely covers the most important bases. Under this definition I wouldn’t hesitate to label Malevich’s Black Square art. Long story short Malevich wanted to make a painting that wasn’t “of” something. He didn’t want to show the beauty of some thing, he wanted to be completely original and show the beauty of an ideal shape. Now this definitely satisfies the criterion of being imaginative, though a little weak on the technical skill part. Is a square beautiful though? For example look at Piet Mondrian’s Broadway Boogie Woogie. It’s a collection of squares and it’s got colors so it’s not that far off. Is it considered beautiful? Follow up question – do you believe pictures of the New York City grid is beautiful? My point here is that incredible technical skill is not a prerequisite for having beautiful artwork. Another angle to look at this is how nature can be beautiful. A flower is beautiful not because of the incredible technical skill of it’s DNA. It just is beautiful because you like it.

So modern art is in fact art. Cool but what caused that change from making such “beautiful” works like David to throwing a dead shark in some preservative? I’ve heard that it accelerated with the introduction of the camera. Art could no longer just recreate reality, it needed to innovate and differentiate itself. What became important was the artist’s subjective interpretation of a feeling, moment or idea.

I can’t speak to why you should give a shit about modern art. This brings up the question of why you should give a shit about art at all, but that’s for another time. For me, I always felt like I was missing something important. You’d see people reading deeply into a few squiggles on a canvas and I’d wonder what they’d been smoking. Modern art always had the trappings of legitimacy but it was tough to take it seriously. Once I’d learned a little more about modern art I realized how really it runs parallel to pop culture today, which is at least one way it makes a difference in your life.

Drake’s Hotline Bling was definitely a pop culture moment. I only learned later his music video totally cribbed the work of James Turrell. I’m a pretty big fan of Turrell’s. His most famous works are his Skyspaces, which are small buildings with open skylights. They’re illuminated on the inside with color changing LED’s which contrast with the color of the sky. I think this is a super cool idea which really gets you to focus on the color of the sun where otherwise you wouldn’t.

The art you enjoy every day is influence by modern art. Team Fortress 2 impressed me with it’s creative art style that was incredibly unique even today. Turns out it was influenced by the Precisionists.

Another artist that I discounted was Andy Warhol. In particular I thought copy-pasting the Campbell’s soup can wasn’t really that big a deal. One way of looking at his work is that he was the first to bring an appreciation to the artistic value of advertising and marketing. He bonded high art with the artwork the average person was most likely to come into contact with. This was a remarkably egalitarian view of artwork. I personally don’t think his work is “beautiful” but this relates to the Slate Star Codex post on reading things backwards . Imagine what a sea change it was when Warhol showed that advertising could be artistic.

Cut Piece was a piece by Yoko Ono which I was surprised I appreciated. I didn’t think much of performance art previously, but this was something that was really out there. Ono sat on stage silently while the audience could cut her clothing away. It forced the concept of audience interacting with artist to be explicit. It defined the relationship as potentially destructive and aggressive.

Duchamp’s Fountain was also a work that I would have sneered at but grew to appreciate. My opinion can be summarized by the following quotations from Louise Norton.

“Whether Mr Mutt with his own hands made the fountain or not has no importance. He CHOSE it. He took an ordinary article of life, placed it so that its useful significance disappeared under the new title and point of view – created a new thought for that object.”

And

“The only works of art America has given are her plumbing and her bridges.”

The point of this is that maybe you don’t dislike modern art, you might just dislike certain kinds of it. Keep an open mind and explore some different kinds of modern art.