Two Weeks of Colorizebot - Conclusions and Statistics

by Amir Avni

About two weeks ago we released ColorizeBot to wander around Reddit. This Reddit bot has started by coloring images on r/OldSchoolCool and a day after, spread all over Reddit.

The bot was based on a pre-trained neural network - More information on the netwrok can be found here.

After thousands of images and thousands of comments involving questions, compliments and insults, we decided to gather all the information on ColorizeBot's interactions, and share some of our thoughts and statistics. If you browsed here because you are interested in statistics, roll down to the second part of this post.

Note that in this post we only refer to photos that were uploaded by Reddit users. If you are offended by one of the photos or it is a private photo you would like us to remove please contact us.

Check the full album of a few hundred of photos on Imgur, or just browse here:

Colorizebot Latest Work

We roughly picked around 600 photos out of a few thousand photos the bot had created. A lot of photos were removed such as all the NSFW photos (Sorry...) and also all the non-photos such as comics and screenshots.

Overall it seems the bot does quite an amazing job. We must remember the coloring algorithm was trained using ImageNet, which is a set of 80000+ images. The images were transformed to B&W and then the program was trained to paint them back to the original color comparing its results to the original colors

Since all the photos on ImageNet are high quality photos, the algorithm works best with similar photos. Here's an example of new and high quality photos which were uploaded by Reddit users:

Old photos on the other hand, were taken using old cameras and have different characteristics. In general, the histogram is different, which means that if we test the "gray color" distribution over similar images, with old and new cameras, we will get different results. We still try to find more information about this issue. If we can find a model that represent the differences, we can "re-new" old photos and then they will be colorized in a better way. Also, old photos have been destroyed over time, we found them much more foggy and with stains which also made the colorization process less optimal. Check out this photo for example:

While the colorization is somewhat decent, we cannot ignore the blue stains in the image. If we examine the floor and the wall in the B&W image, we can see the contrast is very wide. The front of the floor is almost white while the end of the floor is almost black, same goes for the wall. The neural network "trained" itself to identify patterns, texture, luminescence, and the relation between them, and then get a decision on the color it will use. If these parameters tend to look different in old photos, the colorization will probably turn out wrong as well.

However, a lot of old photos did get good colors, such these:

Just my intuition, but those images seems to be in good quality and this is probably the reason why the colorization is better.

Another thing we've notice, is that clothes are mostly will not be colorized correctly. The reason for that is probably the vast variety of clothing in the training process, which made the algorithm to tend to color certain clothes in certain color. When we are not sure what was the original clothing color we won't mind if the color is wrong because the coloring seems good anyways, but when it comes to images of soldiers, were we expect their uniform to be green or very dark green, we notice the bad results very clearly, also in new photos.

A cool thing we really liked was testing the colorization of some paintings. In some cases, the painting was so realistic, that coloring it was almost natural:

On other cases, the original painting wasn't B&W, but had unrealistic colors. Any time the bot was summoned to colorize a colored image, it first transformed it to a B&W image and then tried to color it. In the next paintings, you can see the images are very realistic, but the colors aren't. As a result, ColorizeBot coloring the image in a more realistic way:

Some Statistics (Data is Beautiful)

In this part we present some statistics on the bot's activity, just for fun, because we love data! The statistics are updated for today (Aug. 5th 2016, morning).

  • The bot has colored 2555 unique images. By unique image we mean we don't count several colorization on the same post.
  • Another 329 images were colored using private messaging (Which did not take into account in this post).
  • Around 10% (~250 images) were NSFW.

  • Around 33% (~850 images) were comics images, smartphone screenshots or already colored images, even after we specifically requested not to abuse our bot with these kind of images.

  • Over two million views for bot's images on Imgur. Unfortunately we have posted the images anonymously so cannot extract too much data from here.

  • We got a lot of comments. Some were good, some were bad. most of the bad comments were from r/me_irl where people have decided this bot should color every comic they upload. Oh well, here's an visualization of all the words used in the comments and how frequently they were used. This visualization was done using word cloud:

  • Seems like 7am and 3pm UTC were when ColorizeBot was the most active:

  • Top10 and bottom10 comment Karma by channel (Total 21K+):

  • The bot has operated on these 363 subs (Meaning it had colored at least one image on any of these subs):

r/OldSchoolCool , r/pics , r/meirl , r/interestingasfuck , r/creepy , r/lewronggeneration , r/OldSchoolCoolNSFW , r/TheWayWeWere , r/evilbuildings , r/funny , r/blunderyears , r/bigasses , r/Celebs , r/CringeAnarchy , r/Moviesinthemaking , r/FrankOcean , r/meirl , r/4chan , r/gameofthrones , r/WorldofTanks , r/WorldOfWarships , r/civ , r/mildlyinteresting , r/aww , r/MapPorn , r/WTF , r/colorizebot , r/cosplaygirls , r/dankmemes , r/ColorizedHistory , r/simps , r/colorizebwphotos , r/AdviceAnimals , r/celebnsfw , r/vgb , r/DonaldandHobbes , r/RetroFuturism , r/AccidentalRenaissance , r/calvinandhobbes , r/whatisthisthing , r/PhotoshopRequest , r/comics , r/QuotesPorn , r/IASIP , r/FULLCOMMUNISM , r/BikiniBottomTwitter , r/EmiliaClarke , r/trees , r/PropagandaPosters , r/beatles , r/Heavymind , r/spacex , r/nasa , r/pokemongo , r/malelivingspace , r/oldpeoplefacebook , r/awwschwitz , r/subaru , r/socialism , r/EmmaWatson , r/PenmanshipPorn , r/animeirl , r/BlackPeopleTwitter , r/drydockporn , r/aviation , r/Warthunder , r/Frisson , r/awwnime , r/totalwar , r/analog , r/Tinder , r/WarshipPorn , r/misleadingthumbnails , r/DestroyedTanks , r/drawing , r/NoSillySuffix , r/GermanWW2photos , r/skyrim , r/woahdude , r/Colorization , r/HiTMAN , r/nsfw2 , r/NoMansSkyTheGame , r/bois , r/WarplanePorn , r/UIUC , r/TheDonald , r/blackandwhite , r/AnnaFaith , r/ANormalDayInRussia , r/WWIIplanes , r/AlexisRen , r/india , r/MachinePorn , r/Weakpots , r/greece , r/EnoughTrumpSpam , r/GirlsMirin , r/marvelstudios , r/singapore , r/alexandradaddario , r/wwiipics , r/CelebrityJOMaterial , r/minimalism , r/OldSchoolSad , r/itookapicture , r/WeirdWheels , r/test , r/forwardsfromgrandma , r/LSD , r/4PanelCringe , r/arresteddevelopment , r/trypophobia , r/ImGoingToHellForThis , r/Jazz , r/seinfeld , r/Bondage , r/lucypinder , r/OldSchoolCelebs , r/PS4 , r/cats , r/terriblefacebookmemes , r/GoneWildHairy , r/gaming , r/toronto , r/hillaryclinton , r/Firefighting , r/RachelCook , r/bodybuilding , r/candiceswanepoel , r/vintageads , r/195 , r/Audi , r/futanari , r/shittyfoodporn , r/punk , r/thatHappened , r/teenagers , r/NatalieDormer , r/FunnyandSad , r/guns , r/happy , r/McKaylaMaroney , r/bizarrebuildings , r/fivenightsatfreddys , r/EDH , r/starlets , r/StarWars , r/EmmaStone , r/nipslip , r/gamegrumps , r/ichiel , r/Eyebleach , r/Dallas , r/pic , r/badtattoos , r/PetiteGoneWild , r/Sundresses , r/vancouver , r/lastimages , r/manga , r/justneckbeardthings , r/pokemon , r/CombatFootage , r/facepalm , r/gwcumsluts , r/oldmaps , r/ScarlettJohansson , r/Images , r/oddlysatisfying , r/detroitlions , r/Eve , r/movies , r/hardbodies , r/vexillologycirclejerk , r/jordynjones , r/wsgy , r/beards , r/TheDepthsBelow , r/gonewanton , r/Israel , r/MoviePosterPorn , r/OnePunchMan , r/tumblr , r/fo4 , r/privatestudyrooms , r/PrettyGirls , r/TaylorSwiftPictures , r/DaisyRidley , r/chicago , r/canada , r/RoomPorn , r/boston , r/blender , r/StrongAndPowerful , r/DIY , r/NSFWfashion , r/thinspo , r/ComedyCemetery , r/kendalljenner , r/pyrocynical , r/Amd , r/teslamotors , r/annakendrick , r/bertstrips , r/lordqtest , r/LaBeauteFeminine , r/TankPorn , r/Spanking , r/pawg , r/CaraDelevingne , r/DatGuyLirik , r/manchester , r/facebookwins , r/RotMG , r/OSHA , r/Rainbow6 , r/civbattleroyale , r/ColorizeMe , r/ImagesOfThe1910s , r/EvolveGame , r/AlbumArtPorn , r/TheGreatWar , r/worldbuilding , r/paradoxplaza , r/Damnthatsinteresting , r/2meirl4meirl , r/CrappyDesign , r/ImagesOfThe1930s , r/survivor , r/totallynotrobots , r/Pareidolia , r/spaceporn , r/AnythingGoesPics , r/cumshots , r/streetwear , r/oldschool , r/SexyWomanOfTheDay , r/TokyoGhoul , r/eu4 , r/AdrenalinePorn , r/Blowjobs , r/brasil , r/TheSimpsons , r/SelenaGomez , r/sadcringe , r/desktops , r/CelebsExposed , r/Zendaya , r/DotA2 , r/WhiteAndThick , r/WWII , r/vintagejapaneseautos , r/futurama , r/nsfwbw , r/circlebroke2 , r/CozyPlaces , r/anything , r/CineShots , r/Civcraft , r/StrangerThings , r/MSPaintDoodles , r/Celebsreality , r/oldindia , r/HistoryPorn , r/hoi4 , r/menslegs , r/Whatisthis , r/freelancejobs , r/NotTimAndEricPics , r/Metallica , r/VintageSmut , r/RachelAnnYampolsky , r/charlixcx , r/CellShots , r/parrots , r/techsupportgore , r/kpics , r/zootopia , r/BMWS1000RR , r/AlessandraAmbrosio , r/historyboners , r/Kanye , r/Wehrmacht , r/surrealism , r/nsfw , r/ImagesOfThe1940s , r/zeroescapecirclejerk , r/criterion , r/fapsassinations , r/food , r/imagesofthe1960s , r/LadyBoners , r/ADifferentEra , r/sexyhair , r/RATS , r/Models , r/onetruegod , r/Weakendgunnit , r/explainlikeimfive , r/ImagesOfArizona , r/testingbubbled , r/goddesses , r/Gamingcirclejerk , r/bleachshirts , r/TumblrInAction , r/piano , r/TinyAsianTits , r/surrealmemes , r/tanlines , r/Daguerreotypes , r/yvonnestrahovski , r/leaseydoux , r/DeFranco , r/CRH , r/glitchart , r/GetMotivated , r/CelebFakes , r/doodles , r/vintage , r/HumanPorn , r/Rule34LoL , r/rickandmorty , r/girlsontoilets , r/ClassicScreenBeauties , r/lizgillies , r/cincinnatibeer , r/GIRLSundPANZER , r/Sup , r/oldschoolcreepy , r/TheSilphRoad , r/SabrinaCarpenter , r/OldSchoolCoolMusic , r/Romania , r/nicolapeltz , r/modsmodsmodsmods , r/asstastic , r/Colorslash , r/BlackAndWhiteGW , r/gonewild , r/Mariners , r/IcePoseidon , r/nsfwsports , r/CelebrityNipples , r/BellaThorne , r/wtfstockphotos , r/curvy , r/ArianaGrande , r/geography , r/mildlyinfuriating , r/PuzzleAndDragons , r/subredditsarehashtags , r/rupaulsdragrace , r/TrollXChromosomes , r/Seahawks , r/ShittyCar_Mods , r/CODZombies , r/somethingimade , r/Battleborn , r/pcmasterrace , r/norge , r/colorizationrequests , r/Kappa

  • The bot was banned from these 31 subs:

r/4chan , r/Romania , r/imagesofthe1960s , r/ImagesOfThe1800s , r/ImagesOfUSA , r/ImagesOfThe1910s , r/ImagesOfThe1920s , r/ImagesOfThe1930s , r/ImagesOfThe1940s , r/ImagesOfThe1950s , r/ImagesOfMaryland , r/ImagesOfArizona , r/norge , r/creepy , r/Celebs , r/OldSchoolCelebs , r/starlets , r/celebnsfw , r/pics , r/gaming , r/BikiniBottomTwitter , r/WarshipPorn , r/RealGirls , r/MilitaryPorn , r/EarthPorn , r/comics , r/funny , r/fivenightsatfreddys , r/cats , r/GetMotivated , r/DIY

Have more ideas for cool statistics? Lets us know, we will try to gather the data.

As a bonus, I took the bot's algorithm and started testing it on some video-clips. Of course the results aren't perfect since it was trained on photos and not on a batch of photos, but the results are still interesting. Check it out, I'll add more videos when I'll have more time:

The Beatles - I Want To Hold Your Hand

Black - Wonderful life

King Kong Final Scene

Lukas Graham - 7 Years (Created By Ron Zohar)

AA