Are you OK, Cyberpunk? - Transformers diagnosis

Posted Dec 12, 2020 Updated Jan 19, 2025

By Kamil

24 min read

At the end of 2020, after 8 years since announcement, Polish game development studio CDPR released its flag game titled Cyberpunk. A big success of CDPR’s previous game Witcher 3 and their “gamers-first” approach implied CDPR being perceived as a golden child of a gaming industry. CDPR was seen as one of few healthy apples in a basket of rotten fruits. Of course, there was only one emotion towards CDPR – love.

All these raised the expectations towards CDPR’s new game very high. Announcement of Keanu Reeves – persona absolutely loved by the internet – “playing” one of the characters Johnny Silverhand grew the hype to the limits. Studio surpassed 8 mln pre-orders world-wide potentially beating GTA5. Right after launch there was over 1mln people playing it on Steam.

The sweetness unfortunately went together with a little bit of bitterness. During first 24 hours Cyberpunk Reviews on Steam reached only 73%. Given Steam’s binary review system, it means that 27% of people expressed negative feelings about the game. The goal of this post is to figure out what are the reasons behind it via data-driven approach.

What’s the plan?

The plan is rather simple. We will fetch all English reviews generated during first 24 hours after launch from Steam platform via their public API and simple python script. Then we will embed all these reviews (after a simple cleaning step) via Sentence Embedding using Transformers, to finally cluster and visualize it using well-known techniques UMAP + k-means. Then we want to look into our underlying exemplars – we will achieve it by picking few examples that are the closest to our centroids computed by UMAP in our embedded space.

ELI5: we will look into the most common opinions about the game – this will give us a feeling about how it was received “in general” – hopefully exposing some interesting, non-obvious details.

Some technical details – data source / preprocessing / embedding

Steam offers public API that one can use to fetch all the reviews for given game. It is described here. We are after one particular endpoint.

To get all reviews for a game start calling it with

cursor='*'

and then iterate further using cursor that you receive in your response. API response is quite rich, you can see some basic information about the author, timestamp, the review itself + some more. Check out the docs yourself.

Here is the simple python script I used to fetch the data.

Each review will be seen as a set of sentences a person has to say/write about the game. Meaning every review will be tokenized via sent_tokenize from NLTK – basic Natural Language Processing toolkit.

For embedding we will use Sentence Transformers, which is insanely simple to get the embeddings from:

        
      
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('roberta-large-nli-stsb-mean-tokens')
sentence_embeddings = model.encode(sentences)

This will transform our input sentences into points in high-dimensional dense unified space where we can apply clustering techniques.

Please note we use roberta-large-nli-stsb-mean-tokens which is the pre-trained model suitable for Semantic Textual Similarity (which we happen to be after!)

ELI5: we decompose each review into a set of sequences, throw it all into one big bag, and look for groups of semantically similar sentences.

Low-dimensional UMAP embedding for visualization

Vectors that we get from Transformers embedding are sequences of 1024 numbers. Each vector represents a single sentence – it encodes its semantics into a bunch of numbers. All these vectors live in 1024-dimensional space, which is pretty hard to visualize. A popular way of dealing with it is to further reduce data dimensionality to achieve 2D. We are going to follow that path. To get there we will apply a technique called UMAP, which stands for Uniform Manifold Approximation and Projection. Don’t worry about the details too much – all you need to know is that it consumes a bunch of high-dimensional vectors (that we already have) and returns two-dimensional embeddings.

After application of UMAP we get points in 2D.

Because we started from vectors that represent semantics, we hope our 2D embedding has the following property: Points that are close to each other represent sentences of similar meaning. Given that intuition we can sense the results by examining groups of points – clusters.

2-dimensional space of sentences in Steam reviews is depicted below:

(each point is UMAP embedding – using cosine-similarity – of sentences previously embedded via Transformers. Background color represents ratio of positive comments in local semantic neighborhood. Alpha channel reflects points density.)

ELI5: Points above represent single sentences written about the game. Local areas imply similar meaning of sentences. Red clouds represent negative comments in the “area” around – the stronger the color, the higher negative rate. Green clouds, on the other hand, are good things people say about the game. Clusters separation equals different semantics – different meaning.

Later in this post we will look into details of some interesting clusters. For now, just to give some sense of what we are looking at, let’s look at the very left, very green cloud. What is it? It is a cluster of 140 comments expressing joy – to say the least – of petting a cat. Hearts of these people were definitely stolen by that single detail as positive rate in this cluster is 100% – no single negative comment about the cats 🙂

Example central sentences in that cluster are:

F*in amazin and can pet kitty best game

cat go prrr when you pet it

YOU CAN PET THE CATS AND DOGS OMFG!

10/10 I could pet the cat

Also, you can pet the cats in game

We could probably end this post as the main finding is there: petting cats cluster is no longer a rumour – it has been confirmed to exist.

As an appendix then, let’s try to look at the other clusters of review sentences.

Clustering

Clustering is a technique of grouping points together – in our case, we already reduced data dimensionality to 2D so the problem is rather easy now. By visually inspecting the point cloud above – we can already see separable clusters. To get them quickly we will apply basic clustering technique called k-means. The only parameter k-means requires is number of clusters which we blindly guess to be 75. The Industry standard is to shoot / guess parameters until you have satisfying results – then you call it an intuition and consider yourself an AI virtuoso 😉 Another, less significant clue here is that number of clusters will represent level of details we are after – compression rate, maybe. Choosing 75 of them means that we are after rather general – not very detailed – image of what our data contains.

Our clustering technique clustered dense semantically similar regions together. Each cluster is represented by so-called centroid – which one can see as an “imaginary” sentence that is most representative for the cluster. The next step is to look at it – we will do it by looking at the closest neighbors for chosen centroids. We are going to look at three types of clusters:

the most dense clusters – top 5 the most common statements about the game – the main thoughts around it
the most positive clusters – top 5 with the highest positive review ratio, good things people say about the game
the most negative clusters – top 5 with the lowest positive review ratio, heart-breaking statements

To visualize all of them we will show set of example sentences (closest to centroid in given cluster), cluster wordcloud and also histogram of time played in the game by the authors of sentences in that cluster.

Dense clusters

Let’s start with the most popular opinions. It is pretty mixed – people discuss hardware and weapons system in some details plus express their sentiment in a strong manner.

cluster #1 – discussion about hardware, positive sentiment

“Would completely recommend, playable on high settings with 2060, i7 8700, 16GB RAM – Honestly feels like I’m playing a movie, the ridiculous depth and fidelity.”
“Great game Runs great on max settings, 60fps with ray tracing off on a 2070 super”
“I have most things set to High with Ray Tracing on and DLSS set to Auto and it runs pretty smoothly between 45-65 FPS in borderless windowed 1440p with an RTX 2080 and Intel i9-9900K with 32GB of RAM.”
“No issues with a 2080ti and 9700k, max settings and stable 60fps.”
“THIS GAME IS INTENSIVE, I overclocked everything on 4k, got 15fps.”
“Playing on RTX 2070 and Ryzen 3800x, Getting 60fps at 1080p most of the time with settings maxed and ray tracing on medium.”
“Edit 1: Forgot to mention I’m playing 1440p 144hz monitor (game is set to 2k resolution).”
“resolution = 1080p CPU = i9 7920x sits roughly around 25-30% on 4.7ghz (400mhz overclock.)”
“Running only on my CPU only .2% on my GPU.”
“Game runs perfectly fine on my rig, hitting locked 60fps on all Ultra + RT Psycho w/ DLSS Quality at 2560×1440 on a 3080 (check profile for full specs).”
“edit: I’m in good combat rn, no fps drops I got a 2060 with an i7 7700k(which is bad), I have everything on high and DLSS on balanced or quality don’t remember, no bugs there either.”
“Steady 60-70fps in 1440p with Raytracing set to Ultra and DLSS set to quality.”
“That being said I’m running with 3080 FE and 5800X both OC’d, and am getting between 80-100 FPS 1440p with everything set to max other than DLSS.”
“Running RX5700XT, Ryzen 7 3700X and 16gb RAM and the latest drivers, shouldn’t be an issue.”
“Only getting 60-80 FPS with my RTX 3080 and Ryzen 5800X, at 2K High to Ultra settings.”

cluster #2 – opinions about weapons

“Enemies are somewhat OP even in normal, they don’t hurt that much but use powers too much, even when out of sight.”
“Also melee makes no sense, knives do more damage than swords, but jokes on you it doesn’t matter because attacking people in melee doesn’t stagger them so they just shoot you anyway.”
“I haven’t gotten too far in the game so enemies are still somewhat bullet sponges.”
“The combat feels janky, guns are largely unsatisfying and melee feels clunky.”
“When a punch in the face is more damage than a round to the head, you know it’s a problem… Gameplay > Story”
“That is however the only credit I can give since in terms of gameplay while the hacking system is cool they do not really give you great explanations as to how to get crazier weapons or get different types of gear.”
“Not a run and gun.”
“Aim at head and empty clip after clip to kill a single grunt, repeat.”
“Your boss didn’t give you an early game non-lethal sniping weapon.”
“The one that tipped me to get a refund was when the audio for firing the gun completely disappeared.”
“It also doesn’t tell you when guns are burst fire/auto/semi.”
“Also, there are tons of unnecessary smoke effects when firing your weapon.”
“He was there, and shooting enemies with me, but was completely invisible.”
“At one point a guy stuck a gun to my face and my only dialog option was to say basically “no worries” after the fact.”
“I don’t mind hard-to-kill enemies but not when done in this way; everyone is a bullet sponge.”

cluster #3 – 10/10

“I’m only 9 hours in and I’m loving every second of it.”
“10/10, literally the only game I own where I get to play as a trans woman”
“10/10 Would watch myself get cyber surgery in real time again”
“I’ve only had the game for 10 hours as of writing this review, and I plan to come back to this and update it as the game progresses.”
“you can have a small pp 10/10”
“This is just a first impression from the first 10 or so hours of my playtime The game is gorgeous, has an engaging narrative and really plays well into a certain kind of “vibe”.”
“You WILL get spoilers, you will regret it, and nobody will give a f… 10/10”
“10/10 would laugh at T-posing and people walking through stuff again”
“playing for 9 hours feels like watching movie, so entertaining and get myself into the story”
“Within the first ten mins of my corpo prologue I was talking to a character that knew me from years ago talking about things that you did together.”
“spent an hour on character creation 10/10 gameplay”
“Do I need 10 more levels and several more hours of piling scrap to make them anything worth using?”
“I’ve played the game now for 10 hours straight and I think this is the best graphics I’ve ever seen, I thought Red Dead Redemption 2 had the best graphics but now I was really impressed of how good the graphics is!”
“10/10 how can you not like a game with a dick slider”
“Can make smol PP 10/10”

cluster #4 – no love

“This game is trash…”
“This game sucks lmao”
“Damn ruined all the patience I had for the game.”
“shitty game theres gays and trannys”
“Runs like shit plays like shit and the story is shit.”
“THIS GAME CRASING SO BAD.”
“Also heaps of trash everywhere.”
“GAMEPLAY (The Bad): Inventory management is pretty messy.”
“I’ve never seen a worse release for a game.”
“The game is in a poor state right now.”
“The recommended specs are not at all good enough at the moment.”
“This is terribly optimised, even for people with “recommended” specs.”
“I won’t speak for higher end cards, but on launch day the optimization is horrible.”
“The game seems pretty bad right now.”
“Wait a few months, this games bugged like crazy and not worth playing currently”

most dense cluster #5 – love

“Yes, very fun”
“You will definitely enjoy your time here.”
“The prologue was great.”
“Background are decent.”
“But… it’s still very, very good.”
“It’s great!”
“Yea, very cool!”
“My two favorite design choices are: 1.”
“It’s great.”
“Enjoying it a lot.”
“Fricking awesome!!”
“The positives: + Jackie Welles, I like him.”
“Just having an absolute great time!”
“So yeah, overall an awesome experience.”
“Holy fucking awesome.”

Negative clusters

Some portion of heartbreaking statements about the game. Looks like people that gave the most negative opinions were disappointed, went for a refund, complained about game crashing, hoped for improvement after the patch (!), but still did not like the story.

cluster #1 – refund

“So glad Steam gave me a refund.”
“If you are leaning to purchase it, don’t forget about the refund feature steam has.”
“I will refund and give my money to someone that deserves it.”
“Thank goodness for steam refunds!”
“after 44min of playing i went to get a refund”
“Thankfully, I got my refund.”
“My Steam refund request was accepted by Steam.”
“Dropped it, refunded.”
“My first refund of my entire life.”
“Would like my money back please.”
“Thanks for the refund, will be picking it up at a later date.”
“also i refunded it.”
“Getting a refund.”
“Thanks for the refund steam!”
“Edit: Received refund”

cluster #2 – game crashes

“Game crashes immediately on startup.”
“Bad point would be the two crashes I’ve gotten.”
“When I started with corpo, after I sat on the chair, my game crashes.”
“Combat at best felt clunky.”
“A broken unplayable mess, every time I boot the game up I am greeted with “Whoa!”
“Crashes after first cutscene.”
“– Story feels like I’m constantly being fucked.”
“My game keeps on flatlining every time I exit the bar at the start of the game.”
“It covers most of the sub-menus and makes managing inventory and upgrades impossible.”
“Can’t play, crashes on opening trailer.”
“Can’t even start the game – verification code can’t be sent – “please try again, if that doesn’t help just wait until our shitty GOG servers can process your request”.”
“Do they collect crash data for support, BUT OF COURSE.”
“Minus another mark for sound.”
“Initially, I crashed twice after getting to the character creation.”
“Many cutscenes and quest script hooks are broken or unreliable and it really does make the main story less enjoyable.”

cluster #3 – it’s bad but will get better

“Would prolly postpone playing further until more patches rolls out.”
“2/10 wish it wasn’t too late to refund.”
“IT GETS BETTER OVER TIME.”
“I don’t think this is worth a purchase until some patches/updates have been released.”
“Stay away until they maybe optimize it.”
“Don’t recommend to anyone until more patches.”
“Let’s hope something happens in the near future about them.”
“But if you can wait, I wouldn’t recommend it right now until the patches come out.”
“I don’t know, I might wait to play this further next year with more patching (hopefully they patch the heck out of this).”
“I’d recommend holding off for awhile until it can be patched over a few times.”
“You may want to hold off until more patching can be done… Radeon r5700xt: 1440p on Medium.”
“Otherwise wait until it is patched up a bit and more optimized, or you have money for better components.”
“I suggest wait for A LOT of patches to iron out those performance issues.”
“One of the best and I can only see it getting better over time.”
“If you are reading this and thinking about getting it, I think you can wait a bit more until it gets fixed.”

cluster #4 – disappointment

“Some of the worst I’ve seen.”
“Furthermore, the customization was disappointing.”
“But that is also very broken, with delayed and sharp input.”
“This shit is atrocious.”
“However a lot of you are going to be very very disappointed.”
“Frankly, pathetic.”
“Very disappointing launch.”
“Wraiths very mad.”
“I’m pretty disappointed as of now.”
“It makes it really really hard to enjoy.”
“But this has left a really bad taste in my mouth and I cannot express my disappointment.”
“As this is negative for people in my situation I made this a negative recommendation.”
“tbh I was already sad & low key hurt that he died but going to his ofrenda just got my emotions all out bro.”
“I have been waiting for this game since 2011, it hurts to say that this game is a let down.”
“This game was not worth waiting on launch.”

cluster #5 – story is bad

“But so far the writing has been hit-and-miss and some of the story choices are disappointing to me.”
“This game reminds me of Watch_Dogs in a lot of ways but worse.”
“Corpo prologue was interesting narratively but oh my god was it boring otherwise.”
“But this doesn’t give you any rewards like new dialogue options or exp, it’s just more dialogue as far as I have seen.”
“They never have anything interesting to say either, they don’t talk about current events or things that you’ve done, it’s all small talk just there to distract you.”
“The story gives you no time to bond with any of the characters and at some parts will just have you mashing skip.”
“and the rest of the real “prologue” which people are misunderstanding as the prologue is so linear and boring.”
“The story is generic and uninteresting.”
“It feels like Watch Dogs it has mildly interesting gimmicks and storylines but just overall just lacking in soul and in depth.”
“Graphics are good, not great, the city tends to be repetitive, the story is average so far…”
“story telling is bland.”
“Characters are boring.”

Positive clusters

People love Cyberpunk for its amazing graphics, detailed world and all the surprises Night City has to offer. And of course for the ability to pet a cat.

cluster #1 – petting the cats

“You can pet cats 10/10 Game of the Year”
“Can confirm you can pet the cat.”
“YOU CAN PET THE CATS AND DOGS OMFG!”
“I’ll see if we can pet any other animals.”
“Also, the fact that you can pet the cats is highly appreciated.”
“GOTY without a doubt for the next year Reason 1 : cat Reason 2 : you can pet a cat Reason 3 : repeat that again Bonus : JUDY IS HOT”
“I love that you can pet the cat.”
“Hehe You Can Pet Animals 🙂”
“You can pet the cat.”
“My cat approves and if she approves, then I do”
“Best part is when you get to pet the cat.”
“Pet the hairless cat after you augment your eyes!”
“You can pet the animals Best game ever.”
“You can pet the cat in this game So No reason for bad review.”
“But mostly the cat.”

cluster #2 – worth the wait

“It’s not the second coming, but I feel like it has been worth the wait.”
“I’ve waited since 2012 and the wait was worth it to me”
“Pretty happy they took the extra time to iron out the major issues.”
“It was well worth the wait”
“Was absolutely worth every second spent waiting for the release.”
“It was well worth the wait.”
“Worth the wait to chill with Gary The Prophet.”
“It was well worth the wait.”
“The wait was worth it just to pet the street cat.”
“It was well worth the wait.”
“This has been worth the wait.”
“It was well worth the wait.”
“It will be tedious, but it will be worth it in the long run.”
“The wait was worth it, and I definitely recommend it.”
“Damn but it was worth the wait.”

cluster #3 – amazing graphics

“Graphics look amazing to me, I really like the visual style of the game.”
“– The graphics in this game is unlike any other open-world games I have seen.”
“Other than that, the game is amazing, and the graphics are still blowing me away every time I see something new.”
“Those who have old setup, remember this is very complex game (never seen this much complex game before) with very high graphic demand.”
“I’m only between medium and high in graphics settings and it looks great.”
“Game is smooth, graphics are incredible.”
“Amazing graphics on Ultra, fluid, greatly optimized and the story is absolutely killer.”
“This Game Has Better Graphics Than My Life and It’s Breathtaking!”
“Loving the graphics and story.”
“Awesome Game and graphics so amazing!”
“This game is INSANE the graphics are really good”
“This is the most graphics I’ve ever seen in a game.”
“Very serious game with very good graphics”
“The graphics are stellar, the gameplay is nice and the world is massive.”
“Overall great game with amazing graphics.”

cluster #4 – detailed world

“The detail on every aspect is just mind blowing.”
“But the density and how detailed and lively is where it shines.”
“Really immersive world and atmosphere.”
“The details put into every single corner is absolutely amazing.”
“The level of details even in Low, Medium settings is absolutely insane, it’s so vivid.”
“The world feels fully realized and fleshed out.”
“Attention to detail is mind boggling.”
“This piece is rare.”
“Incredible level of detail.”
“Level of detail is breathtaking and textures (when they load) are beautiful.”
“They really did a great job on the SFX detail on this game.”
“There’s sooooo much graffiti even in the most remote alleyways, and each one really looks like the artist put a lot of love into it.”
“Pros: ✧ It is beyond immersive, you are thrown into a wonderfully detailed world which feels real.”
“World is so amazingly detailed.”
“Amazingly immersive and beautiful.”

cluster #5 – Night City is awesome

“There is just so much to do around Night City, that when you first start out, it’s a bit overwhelming trying to figure everything out.”
“I quickly found myself getting caught up in the countless events — both huge and small — that were happening in Night City at any given time.”
“There’s so many things to explore and to see in Night City.”
“Night City looks stunning, feels like a genuine big city, roads are narrow, the characters walking around are versatile and nothing looks the same.”
“So much fun I messed around and pulled an all nighter”
“A well designed open world, interesting main story (although I am still very early) and a thoroughly thought out background of Night City.”
“Once you’re past the opening mission and in the city expect another hit in performance.”
“Wandering around Night City is so immersive, graphically looks amazing and A TON to do to keep you busy.”
“The gameplay is solid, the aesthetics are great, and there’s a lot to do and see in Night City.”
“The scale and scope of Night City is simply stunning, and driving around marveling at everything it has to offer – from the massive skyscrapers to the immaculately detailed in-world advertisements that plaster them – is frequently rewarding on its own.”
“The sheer amount of things in Night City makes GTA V pale in comparison.”
“Night City IS a fleshed out city, Night City has A LOT to do.”
“Night City is incredibly lifelike, detailed, and brimming with things to do and places to explore.”
“There’s so much to do and see in Night City.”
“The action, script, art, music and Night City is just a place to get lost for hours”

Final thoughts

The intention of this post was to expose what people think about Cyberpunk 24h after release. Of course, sentiment towards the game changes over time, and first responders are often those that managed to exit playing quickly – one way or another. Naturally then, sentiment may be biased. Therefore it is not a good idea to build your opinion about the game based on what you have seen here. The focus here is on first 24 hours – one needs to keep it in mind.

It looks like first impression about the game is very emotional – people love the game or hate it, some complaints have been made about the story and weapons system, but there is big love towards the Night City and amazing graphics. There seems to be hope about the future patches but also disappointment – very mixed.

At the time of ending this post, Cyberpunk is reaching 80% positive reviews on Steam – it is getting better. And for me, I focused on writing this piece too much, so it’s time for some rest, playing the game and trying to build my own opinion 🙂

Clustering, Data manipulation

This post is licensed under CC BY 4.0 by the author.