Building

The Idea

My core idea was very simple: find spelling mistakes in websites. This is a relatively straightforward problem and with the help of latest LLM models (which are very good at finding spelling mistakes) I should be able to create a good enough solution. In case this gets famous, I could probably expand into more areas.

The Polish

Just the core idea was very simple to implement, so I wanted on rest of the “product” to be really good looking and polished, something which I have vehemently avoided throughout my career working primarily on the backend. My mind was constantly thinking about offloading this task to some freelancer on Upwork so I can work on the “core” features, but then I forced myself to actually work on the product as that was my initial goal: make a good looking and usable product, not just a highly scalable LLM powered spell checking engine written in Go.

Some of these non-core functionalities I decided on were:

• Taking a screenshot of the problematic section on the page and highlighting the mistakes with a red underline.
• Generating suggestions/corrections via LLM.
• Asking LLM to assign a severity score to each mistake.
• A simple shareable report URL which can be sent to the user.
• Full blown user management / dashboard to track, schedule, and manage scans.

Implementation

I heavily utilised Cursor for designing and coding the application. I chose NextJS as the development framework, as it was completely new to me. Having the same codebase for both the frontend+backend with all the bells and whistles was pretty enticing. I rented a very cheap VPS (5 euros per month) from Hetzner and after some experimentation decided to use PM2 for serving the NextJS application. For deployment, I utilised Github Actions to deploy to the VPS after merging to the main branch:

The other components/libs used were:

• Sqlite
  • for storing users, scan results, credits, etc.
• Redis (running locally, via systemd)
  • For caching the DOM tree of the pages and LLM requests.
• Clerk
  • For Authentication, “Sign In Via Google”
• Puppeteer
  • For extracting text from a website, highlighting errors, and more.
• Dodo Payments
  • MoR provider based in India for handling payments.
• TailwindCSS
  • I have to admit, 99% of tailwind was written by Cursor. Just CMD+K the div and say how you want it to look.

Issues/Roadblocks (possible skill issue)

The NextJS ecosystem at that time (around Dec 2024) seems to be heavily inclined towards using PaaS tools like Vercel, Render, etc. The documentation / guides for deploying a simple application on a VPS were not great (comparing to something like Ruby on Rails).

Some other issues I faced:

• API Support: It feels like NextJS is targeted more towards web development instead of just pure API development. My initial plan was to utilise an API in the frontend and use the same API to expose to customers who may want to use the product directly via API. Implementing this is possible, but there is a lot of duplicate code in each API handler (for authentication, context setting, permissions, etc).
• Documentation: Several times I ended up spending time in reading and writing code which was incorrect as I was referring to an incorrect “Router” implementation. Some of the google searches go to the “App Router” docs and some of them go into the “Page Router” docs. After days, I made it a habit to first check if I am reading the correct documentation.
• Logging in NextJS: Once you build a production bundle, the app stops logging the requests/responses. Some custom loggers/middleware updates have to be done in order to get the logging back.
• Middleware Runtimes: The runtime for middleware is “different”, some node packages can be used and some cannot. I did not find the middleware usage intuitive and was constantly confused on which kind of params to accept in the method.
• Related to above: I felt that Cursor sometimes was creating code which was in the incorrect “Router” mode. And the errors did not help much this.

“Launch”, “Growth Hacking” and Feedback

I spent a lot of time on reading and watching videos on “Launching” and “Growth Marketing”. It seems like Product Hunt was the place to go, and then some newer alternatives like Uneed, Peerlist. I created a twitter account for my product. My plan for marketing was to scan famous websites for spelling mistakes and then tag them in a tweet -> they would notice it and possibly reply with a “thanks” -> and this way I will get visibility and then my product will spread by “word of mouth” and soon I will be cashing in dollars ! :-D That was my plan atleast..

Twitter

I created a new twitter account and I am pretty sure it was/is probably under shadowban, I did not get a single reply, I was finding genuine mistakes in landing pages, docs and more. Continuing further was the hardest, and with no feedback, I was not motivated to continue. So as is pretty common, I thought I need to build more features :-D , so I spent another 2-3 days quickly creating a chrome extensions to allow scanning webpages without coming to my website. I also created a blog section and wrote one blog which I thought would definitely gain some traction.

But all this was not generating a single visit on my website. I had a free no sign-up required scanning service, still no one was scanning their website. I then started DMing indiehackers with issues found on their website, and luckily I got some replies thanking me (but no signups). I then reached out to builders with really small following < 1k, hoping they would be more responsive, but that wasn’t the case.

Reddit

After some days I got an idea to do a “free page scanning” post on one of the sub reddits for indiehackers and builders. That post got pretty good response, I scanned close to 30 websites. The product was working, the people were happy with the issues it was finding but again no one actually signed up :-(

Eventually I posted again on Reddit to get some guidance around the product and how to launch it. I got some really good and constructive feedback. The common theme here was “vitamin vs painkiller” analogy, almost everyone felt that this product is a vitamin and not a painkiller. And they were right, I found several landing pages with some mistakes, but those were not “critical” mistakes in order to catch anyone’s attention. Most of the product on a baseline had good enough “copy” (that’s what they call the “text” in the marketing world).

Product Hunt and The Growth Hacker Call

Without any other option in sight, I decided to launch on Product Hunt (and similar alternatives). That’s when I realised the sheer amount of products which are being launched everyday…do people use these ? Are there so many people searching for new products everyday ? voting, commenting on products, giving feedback ?

It was pretty overwhelming to see big well funded companies launching products everyday. How will I compete with them ? Most of the products already had tons of users, very rarely were there products from unknown names / solo devs or with zero users. Some products were being launched again after a launch done an year ago. The only thing I could understand from all this was the need for someone to “hunt” your product. If one of these product hunters hunt your product (According to PH) there is a good chance that more people will notice you. That’s exactly what I wanted, I wanted to be noticed !

So I reached out to one very famous growth hacker / product hunter on LinkedIn and we connected over a call. He gave some good inputs, but eventually it came down to the “offerings”. They had different packages, under some packages they would create high quality Reddit posts from high Karma users to boost visibility and under some they would completely take over your marketing (SEO, blogs, etc), other ones would provide high quality backlinks to the website. There were lots of “tactics” under this so called “Growth Hacking” strategy and according to him that’s the only way to get a new unknown product out in the world.

Vibe Coding Bike Shedding

After the call with the growth hacker, I decided it’s time to make the product really good, I won’t fall for these cheap growth tactics (not actually cheap, they cost $$$).

So anyways I did not stop at just making a basic Chrome Extension, I went on to create screenshots, demo videos, and logo for the extension.

I did some more iterations on the landing page, I thought I am not getting users because the feature descriptions are missing nice logos or maybe I am missing an FAQ section or maybe my pricing page is not looking good on a mobile device..

In a non-AI world, I would not have spent so much time on these useless improvements, as it takes a lot of time to do changes on the UI (for me), but armed with a paid 1 year Cursor plan, I was not going to stop..

Basically with frontend there is a lot of trial and error involved to get things to look how you imagine it in your head. That’s not the case for me when writing non-web code (like an API endpoint, CLI, script, SQL, etc), you sort of know the exact way things will work (barring some non determinism due to infra related issues for instance). Maybe experienced frontend developers can just write tailwindCSS code and it works the first time in the browser, but for me it’s a lot of trial and error (even with the help from Cursor).

But because of Cursor, I was just acting like a frantic product manager asking my dev team (me) to do random changes on the frontend. Like create a toggle for dark mode etc, I thought these things were easy to implement, just change some CSS variables and BOOM now your entire website is in dark mode… But no, that’s not how it works..

I was going to dive into A/B testing my landing page but thankfully I stopped and then I did a course on SEO with NextJS :-) . I created dedicated pages, like “free landing page scanner”.

I integrated OpenGraph meta tags to create link previews for reports, blogs and other sections of the page. I wanted my imaginary users to have a good experience when they would share my reports in their non existing company Slack channels… All this because of Cursor, it fed my ADHD all day everyday and kept my dopamine receptors fried.

I scanned some more websites and sent cold emails to some of the founders/agencies who were building in public, in order to get their attention. I create a new LinkedIn account and reached out to some web dev agency owners, but none replied. I spent a lot of time to go through their projects, find their org structure, and determine the correct people to reach out to.

At one point I had to resist the urge to automate the entire chain:

1. get newly launched websites from ProductHunt.
2. Scan their landing pages.
3. If there are any issues found, create a publicly shareable link.
4. Fetch email ids from Apollo or similar websites.
5. Send them a personalised email with the link to their report.
   • utilising LLMs for create the mail content etc.
6. Add tracking params to these links and send an alert if someone visits / creates an account etc.

I was going to vibe code this away, but then I stopped.

Giving Up

I did a “Show HN” on HackerNews as one last attempt to get some traction and as you might have guessed, it did not get any attention. I did not want to spend more time on marketing the product, as all those felt shots on the dark.

My learnings from this project has reinforced what I knew theoretically from countless other blogs posts and tips:

• Building a product is not the hard part in most cases, it’s getting users to use it and even harder: getting users to pay for it.
• So get users first before building anything. I don’t have any answer or secret formula for this. Most of the times the users are not searching for a product. It depends on a lot of factors.
• Do not build something which you will not use yourself.

Open Sourcing

After making exactly 0 dollars (and spending around 20 dollars on domain, infra & LLM tokens), I decided to stop working on it further. On the bright side I learnt a lot of things around web development, UI/UX and marketing. I never had any big hopes to begin with, so I did not feel like some kind of failure at the end of this all. If this would have taken off, I would be really happy with it. If you are interested here is the Github Repo: https://github.com/pulkitsharma07/spelltastic.io and the website is live here: https://spelltastic.io.

If you plan to build something using this or want to discuss some other ideas, please reach out to me !

Is this Blog part of the growth strategy for the product ?

Good point ! But, No. This post is not in any way to get any growth for my product. I am writing this blog to document my journey and learnings from this project, writing a retrospective post at the end of it all was always my goal. If anything, this blog post is a growth strategy for my personal brand, so that people can know the kind of things I have worked on / built / regretted and my thought process behind those. This blog post is being written to close the loop on this entire thing.

A little background, For those who are not aware Cursor is an AI-first fork of the very popular Visual Studio Code editor. More info here. I guess I am late to the party, but I never really gave any of these AI IDEs a shot…until now.

A New Codebase

At work, we were exploring approaches to quickly spawn up nodes in one of our K8s cluster. The Windows EC2 machines in particular take a looong time to boot up ( > 15 minutes sometimes). Although AWS provides Windows FastLaunch, but it has limitations on how frequently you can launch new machines and the boot up time was still quite high for our usecase. In k8s, the component which spawns new nodes when needed is the autoscaler. It is written in Golang (as most of the K8s codebase) and calls the underlying cloud provider APIs (AWS, GCP, Azure etc) to spawn new nodes when needed, i.e. when there are pending pods in the cluster. Another well known auto-scaling solution for K8s is Karpenter.

I had proposed an idea to the team to patch autoscaler for our custom usecase. I had never contributed to any K8s internals before, so all of this was pretty new to me.

So, there were three approaches for me to move ahead:

• I take some time to understand the codebase and find the relevant methods that I can modify.
• Or use some AI flavor-of-the-month tool to aid in the same.
• Or try to play whack-a-mole by searching random terms relevant to my task, something very common if you are into reverse engineering.

Starting with the Old Skool way

I went ahead with the third approach as sometimes it gets the job done real fast. A Good ol full-text search using ag over the entire codebase with some keywords which I felt could be related to my task: scaleup, AddNode, SpawnNode, etc.

Eventually, I found some interesting code areas, so I went into VSCode to understand the code flow further. Using the Go To Definition and Go To References bindings I was able to quickly get a good overview of the call hierarchy and started to develop some mental image of the code base. But I was still not sure I was looking at the correct code area for the thing I wanted to achieve, after 10-15 minutes I eventually noticed ../oci/.. in one of the file paths I was checking… Apparently, I had spent the last 30 minutes reading code that was completely irrelevant to my use-case, here oci = Oracle Cloud Infrastructure. While going blind into a new codebase the file path may sometimes convey more information than the file contents…

I then looked into the codebase under the ../aws/.. directory and eventually, I found a relevant method that seemed to be something related to what I wanted to do.. but I still wasn’t sure, the method was more than 100 lines long.

I tried to eyeball sections of the code, looking at just variable names and function names to quickly develop some crude understanding of the flow. Then I thought… AI ….Why not use use the AI magic here, we have tools for this now. Since my Github Copilot trial had expired, I wanted to give Cursor a shot.

Enter: Cursor

Installing Cursor was blazing first, I opened the codebase, and all of my existing VSCode extensions were installed automatically and I was already in Vim mode, nice. I quickly opened the file I was checking out earlier and went straight to the function. And that’s when I noticed the Cursor prompt: Cmd+L to generate, Cmd+K to chat, without thinking twice I selected the entire method in visual mode -> cmd+K -> "Explain this and let me know if this is the right place to do <my task>" .. almost instantly I got a summarized version of that 100 line method and I got confirmation from Claude (the underlying LLM) that I was on the right track.

Next, I wanted to check if Cursor understands the call hierarchy of this method and eventually how this method is invoked by the autoscaler. I am not sure how many tokens it consumed, but yes it did give me some information but it was not exactly correct, there were some methods that it mentioned that were not actually present in the codebase. But I was not expecting it to be correct in the first place. Most of the things it mentioned were correct and now I had a very basic understanding of the scale-up phase of the autoscaler.

On this day, Cursor had saved me around 30-40 minutes of dev effort. The rest of the coding had to be done manually, as I had to make changes in multiple files (update interfaces, change method signatures, etc), I was not really sure how to prompt Cursor for these use cases. Plus the overall codebase was quite big, so I am not sure how would Cursor do RAG just based on my prompts. Maybe I could provide a detailed prompt with which it could select the relevant files. But I felt it would be faster (and less error-prone) to just do those changes manually as I knew what I had to do.

Plus Go being a language with excellent VSCode support (and documentation), writing code in it is always a breeze.

PowerShelling with Cursor

To ensure my proposal around patching autoscaler works, I had to make sure that some interactions between the autoscaler and Windows EC2 node work properly. After some chatgpting and googling I was now ready to tweak the userdata script for Windows (generated by EKSctl). This userdata script configures the EC2 machine to join the K8s cluster on boot (and also does some config related tasks/cleanup). For Windows, this script was written in PowerShell, and this is when I became a fan of Cursor.

Programming in Natural Language

This time I utilized the Cmd+L shortcut in Cursor, and I just gave it instructions on things I wanted to do. I was not sure how to create systemd like services that run on reboot in Windows, so I just asked Cursor to do that. I was not sure if powershell even supported functions, So I asked Cursor to just create a function for logging and call it in relevant places. It generated the function and also updated the script to call that function.

Not having to do the loop of: goto browser, goto chatgpt/google/stackoverflow, type query, read, select the relevant part of code (if any), copy, paste in editor, refactor/rename variables… was a real time saver. Especially with PowerShell which I had zero experience of.

Seeing Cursor Think

Another feature I loved about Cursor is the way it processes large chunks of code, it highlights each line as it is processing it, editing it only if it is required to do so. Instead of dumping your entire code into an LLM and replacing it with the response, it seems Cursor is doing something more “intelligent”. It goes over each line (or batches of lines) and highlights it neatly as it is doing so. It suggests the edit as a git diff which you can apply or reject… and then you continue a conversation while it keeps track of the diffs internally.

Less tech debt with higher speed

This reads like an ad, but using Cursor really helped me to reduce tech debt in my code. Traditionally, I would write copious amounts of TODOs in my code, as I don’t find it relevant to spend time on things that are not actually making a noticeable impact in the short term. With Cursor, instead of leaving TODOs in the codebase, I can just ask it to do it. For example, I asked it to Move all logging behind an environment variable named DEBUG.

At this point, I had easily saved about 5-6 hours of effort (and pain) in writing Powershell scripts and corner cases around reboots, string escaping, and exit codes. This saving may seem small, but I think it helped me deliver the final implementation 2-3 days earlier than planned, which is a lot since I had planned to deliver everything under two weeks.

Better Readme

One unexpected win for Cursor was writing documentation(README.md). I like writing detailed documentation around anything I work on. I never really thought about how predictable or “LLMable” this task is. With just a few words Cursor (or rather Claude) was able to suggest what I wanted to write. It felt stupid to not accept the suggestions. Of course, some suggestions were wrong, I can just write those things, I am not expecting it to complete the entire documentation on its own. Overall I saved a lot of time writing the readme and I re-invested this saved time into it again to make the documentation even better.

Final Thoughts

Having AI integration right into the IDE is a game changer. Dealing with “text” is what LLMs are great at and coding is just “text” data, so IMO AI will have a massive impact on coding (especially via these AI-first IDEs). I don’t think this positively about tools like Devin which claim to do pretty much anything related to coding. I am willing to bet that tools that improve humans will actually thrive more than the tools that aim to replace humans.

Using Cursor was a massive productivity boost for me which I had not really anticipated (even after using ChatGPT extensively), and I am happy to recommend it to any developer who has considerable coding experience. As for new developers, please stick to non-AI tools for now as it can really hinder your learning experience.

While reading “The Three-Body Problem”, I developed this habit of keeping a small notebook with me to jot down key moments of a chapter. The purist inside me wanted to fight back and force myself to remember everything. Taking notes felt too “academic”.. but then I just did what I felt was making things more enjoyable rather than do something to cater to the perfectionist inside me… so I took notes.. and I also read them before I picked up the book again.

I found it to be immensely helpful to keep track of what was happening and saved myself from unnecessary cognitive load of remembering things in realtime (while reading). I followed the same approach with Permutation City.

Why you should read Permutation City ? (No Spoilers)

This was my second sci-fi book. There were times where I had to keep the book down and just re-read and try to grasp what the author was conveying. These challenging bits were the most interesting to me, and it also lead me down to discover the FAQ of the book hosted online by the author (Greg Egan).

You should read this book:

• If you have done some work around creating games or simulations.
• If you are interested in learning about recursion, fractals, etc
  • The book doesn’t exactly touch on these but if you are interested in these things, the concepts covered in the book might be of interest to you as well.
• At times you question the nature of consciousness / reality.

If you have not read this book, thank you for reading this blog post, you can leave now

For People Who Have Read this Book (Heavy Spoilers)

!! Please skip this section if you are planning to read the book. !!

What I liked

Experiments on the Copies

I think the best part of the book were the experiments done by Paul Durham on his copy. Especially the very first set of experiment mentioned in the book where the copy is unable to escape the environment.

Public Compute Capacity

The way people can bid for public compute capacity gave a very AWS Spot instancy feeling.

Dealing with both micro and macro simulations

While personally thinking about simulating reality, the main question we often face is: how good the simulation should be ?

• (micro-simulation) Should it even simulate at the level of planck length, i.e. simulate atomic and sub-atomic particles and their interactions ? Perhaps also dive into quantum interactions ? OR
• (macro-simulation) Should it stick to a level of reality which we normally perceive in our daily life with our five senses. Which is a kind of “distance culling” approach applied to our reality.

This book attempts to deal with both sets of these simulation paradigms instead of just picking one over another. Hypothetically, due to the book belonging in the “sci-fi” category, the author could have easily leveraged that freedom and claim a working micro-simulation which renders a complete believable reality on all the levels (perceived and non-perceived). But instead the author proposed a somewhat “humble” and crude set of simulations, with their own set of interesting limitations and workarounds. These imperfect simulations and software drop you in between the action of figuring things out and thinking about various improvements / shortcoming of the system overall. These were the best moments for me while reading this book.

Hiding a program inside a program

This was one of the most important plot points for me. When I first stumbled upon this in the book, I thought of various ways to achieve this.. Compression ? Some kind of AST mish-mash magic ? But what was proposed in the book was even cleverer: extra dimensions. To put is simply use 3D coordinates for representing 2D data, and “hiding” the extra information in the unused 3rd dimension of the vectors. Unfortunately the book never really expanded upon this concept. Instead this was used to convey a story of a couple, which I honestly found a bit boring. (More on that in the section below) I have plans to create a small working demo for some simple ruby scripts. But I am pretty sure I am underestimating the efforts involved here.

MITM detection using change in latency

I am not sure about the exact state of technology (in our real world) when this book came out (1994), so I think this could be an actual implementable novel idea, this was my first time reading about this and it makes complete sense to me. This is the way Paul finds out that there are some people intercepting his network traffic. Because of the potential additional hop in the interceptor’s network card, he notices a change in the latency numbers. I wonder how realistic this would be to use in our current life today ? I guess not, because network paths keep on constantly shifting between two IPs on the internet plus a person doing MITM can mirror the entire set of bytes and then process them instead of actually intercepting the packets. For example, in the case of snooping on optical fibers, one can bend and decode the light “leaking” out of the cable. I wonder if this would impact any timings (or error correction schemes) on the receiver’s or sender’s end.

The chapter names

Midway through the book, while reading about the book on the internet, I got to know that the chapter names were all permutations of each other :-D This was a really nice easter egg to discover, and ofcourse I felt dumb af discovering about this.

What Could be Better

Although it is one of the best books I have ever read, I couldn’t help but list down few things which I would expand more upon if I were to contribute to the writing process of this book.

Dust Theory

My understanding of dust theory: consciousness can be rendered across disconnected events in time (i.e. non-linearly) AND disconnected points in space.

As a thought experiment this is definitely intriguing, but this leads to numerous other possibilities:

• How many such configurations are there ?
• Is each possible permutation of reality/time rendering a different version of a universe ?
• Are these universes stable ? How many of these configurations are pure noise ?
• With theoretically infinite amount of possible permutations, any possibility can be true. akin to the finite monkey theorem.
• Hence, you do not have to worry about whether you service or your clone survives or whatever happens in the past/present/future.

Permutation City itself

The design of the city itself was never really touched upon. I wanted to know more about Carter, the architect of the City. As the booked is name after this City, I wanted to know more about how it was made, rendered, hidden, etc. How Paul reached out to Carter ? Did they brainstorm about different ideas ? The details about how they were hiding some set of people in the city were very interesting (covered in the “hiding a program inside a program” point above), but then again it was never expanded upon. My understanding was that these characters were supposed to invoke some deep philosophical thoughts in the reader about love ? or life ? or our interests ? It never really struck me as profound as some other philosophical topics covered in the book.

Limited Compute Power

In the beginning there is one company which outbids everyone to buy significant amount of compute, I may be wrong, but I never quite understood what that company did with the compute power, was it just the hydropower project ? . Who were the people behind it ? What was the actual purpose ? just to run simulations for a hydropower plant ?

Closing Thoughts

I can go on about what I liked and did not like about the book, but those are just my personal opinions based on my comprehension abilities (and memory). Needless, to say this is the best book I have ever read. Scifi books like this provokes thought as well as inspires me to write. I think this quote from a reader from r/gregegan gave my good direction to think in:

Permutation City was my first Egan read and I can only awe at the ideas and their execution. Interpretation has never seemed necessary for me to enjoy the novel.

acting this way is a metaphor for something rather than an explanation for the reader to decipher

Need to balance and understand: reduction, fiction, metaphor, emotions

After experimenting with self hosting the blog on my rpi, I have moved the hosting back to Github pages. One of the main reason has been the feeling that something was always “ON” in my home. Usually when I leave the house, I like to turn everything OFF. I don’t like the feeling of keeping the Rpi constantly running for a blog which barely has any visitors. I feel that is a waste of resources. The Rpi was not doing anything compute heavy, I had no use case of a proper backend (the blog was still served using jekyll serve), so I did not find any need to continue using the Rpi, felt like an overkill to be honest.

A lot has changed since 2017, when I first started using Github pages for hosting this blog. Github Actions is now a mature product with full blown support for Jekyll. Migrating from Cloudflare to GH Pages for this domain (pulkit.cc) was straightforward. Since I had originally purchased this domain from Porkbun + I had no particular use of Cloudflare anymore, I decided to move the Nameservers back to Porkbun from Cloudflare.

After that I updated the A records to point to Github Pages’ servers. There was some errors in auto provisioning of the SSL certs (on Github’s end), which took a lot of time to debug. I was able to resolve it by adding a CNAME record for www which points to the original github pages domain: pulkitsharma07.github.io, all thanks to a random medium blog.

I removed the extra gimmicky stuff I had added in my about page (It used to print the Rpi temperature/uptime). Also removed the analytics around this blog. Sometime later I might add discourse comments to it.

What is the purpose of this blog ?

Reading my about page induced a feeling if cringe. And this happens every time. The longer the gap in reading my about page, the greater the cringe. I am still not fully sure about the purpose of this blog. My first post went to HN front page, and then 2 more after that. While writing all those I was pretty confident that these will get to the front page, as these are posts which I would have upvoted as a reader on HN. And since then while writing any post, I wanted to write it for the HN audience. I wanted every post to “mean” something, to be HN worthy…I had and have many things I wanted to write, but I just couldn’t because I did not want to write about half baked, personal, stupid, unrelated, non-technical things… but I guess I should change that and not think too much..

• Make use of the Raspberry Pi I had laying around.
• Experiment with some DevOps stuff.
• Feels cool to have a server running 24x7 in your home.

Architecture

Wanted to keep this as simple as possible.

• The Rpi is hooked directly into my wifi router using an ethernet cable.
  • The router is configured to assign a static private IP to the Rpi.
• Cloudflare Tunnels to expose the Rpi to the internet quite securely.
  • this is completely free ! (as of writing this)
  • More details explained in the Software section below.

Pure vs Impure Self-Hosting

I understand that relying on a third-party service like Cloudflare, is not actually self-hosting of the purest form. But I am okay with this impurity based on the time saved and features provided.

Hardware

• Raspberry Pi 4 Model B Rev 1.2
  • Has 4GB of RAM and 4 core 1.8 Ghz CPU
  • Provisioned to run Raspbian OS (64-bit)
    • I am able to run almost all arm64 docker images on the pi.
  • 32 GB sdcard for persistent storage (SanDisk Extreme 32GB microSDHC UHS-3 Card)
• Argon ONE V2 Case
  • Integrated fan really helps in keeping temperatures under control
    • Using argonone-config I can map fan speeds to different CPU temperatures.
  • Prevents dust build up
  • All ports are neatly accessible on one side

Software

Development

I wanted to setup a development environment similar to the setup on my work Macbook. Here are few things:

• Shell: zsh
• Editor/IDE: neovim (Specifically Astronvim)
  • I had to compile neovim from source because there were no ARM v7 releases for that.
• tmux
  • Pretty much standard to allow long running shell sessions.
  • Can have native iTerm experience with tmux -CC. More info here

Hosting

• Cloudflare Tunnels do the heavy lifting for exposing the pi to the internet.
  • A daemon runs on the pi which opens a persistent connection with cloudflare infra. The cloudflare infra then receives requests from the internet and forwards them onto your pi via the daemon.
  • You can expose multiple services via a single tunnel and also restrict public access via authentication.
  • Watch this for more info.
  • A SystemD service unit file spawns the daemon on every boot.
• Docker
  • I am running all my containers with --restart=unless-stopped. This will auto run the containers after boot. Check this for more info.
• Deployment
  • I run nginx:alpine container which has the blog’s codebase mounted as a volume during runtime.
  • Since this is a static website, I just go into the directory and do a git pull and the new version becomes active.

Monitoring and Analytics

• Plausible Analytics running on the Pi to gather web analytics.
• n8n is setup to monitor services and run other workflows
  • This is integrated with my Telegram account which acts as a central place to receive notifications
  • I plan to add more workflows in the future for personal projects here.
• Cloudflare Tunnel Notifications
  • This is a neat feature where you will get notified if pi loses internet connectivity.
• For metrics I run Prometheus on the pi to collect them with a retention of 30days
  • It can collect the pi metrics (temperature, cpu, ram, etc) and also collect metrics from the docker daemon about running containers.
  • Once I am able to get the hang of PromQL, I will be Using Alerts.
• Also I have a small script which periodically updates my about page with my server’s stats.

Future Work

Obviously the things documented in the blog are just the current state and these things will keep on evolving. Here are few things I have in mind:

• Setup Grafana to create dashboards from prometheus metrics.
• Backups (probably using restic)
• Related to the above, I thing I would like to look into is automating provisioning of the pi server(s) from scratch.
  • IMO the playbook or recipe based approaches are difficult to test from scratch, so I would prefer some “image” based approach (where I can change hostname etc).
• Setup log2ram to buffer writes to the sdcard.

A look into why most of the Hacktoberfest PRs are “manufactured” in India

Before reading the post

Now that I have mentioned the name of a country in the title, please read the following points:

• I am an Indian and have been living in the country since my birth.
• I do not claim to be right on all of this and there are many anecdotal generalisations here. Somethings are more like human problems instead of Indian problems but just the sheer number of people here puts India on everyone’s radar.
• This post is NOT to denigrate India. It is definitely critical of some of its social and psychological aspects. As a proof for that I want to mention that I am writing a series on the wisdom of ancient Indian philosophical ideas here: Source [0], just so you know that I am not writing all my posts with the sole intention to show what is “wrong” with India. We should be brave enough to face the good and bad of it.
• It is a shame that I need to write this header in the first place. But the reality is with or without this, people will still get offended.

Index

• What
• How
• Why
  • The Signalling Problem
  • The Jugaad Mentality
  • Computer Science Education in India
  • But why for Hacktoberfest?
  • The problem with high population density
  • Hierarchy of Needs
• Closing Thoughts
  • Probable Future
  • Improbable Future

What

As most of you may know, Hacktoberfest started on 1st October 2020. And as it with most things in human life, people came with different ways to hack hacktoberfest to get the free t-shirt.

Now apart from the sheer number of spam PRs which were opened, copious amount of time is being spent by the comparatively small number of open source maintainers who have comment, close and label each individual PR.

I became aware of this happening from this post, and in a days time several more posts popped up. I got motivated (triggered?) to write this post when I saw the following on HN:

After some investigation, I found the original claim to be true, more than 60% of the spam PRs are coming from one country: India and that is just 1 day after hacktoberfest started, I am sure the percentage will increase as days go by (unless DigitalOcean does something).

How

This post talks about one Indian YouTube channel which apparently told people to open spammy PRs. After I looked around some people were claiming he was falsely accused, anyways the video is deleted now, so I can’t comment on that.

Anyways, it didn’t take long and I was able to find few more Indian YouTubers who were pretty directly promoting ways to open PRs to get those sweet T-Shirts. One created his own repo and encouraged people to google different programs and copy-paste them into a new PR against the empty repo.

I do not want to focus on the How, you will find find plenty videos/blogs on it, I am sure some Indian YouTuber has started working on creating a video on it.

Why

One of the simplest explanation is that, DigitalOcean should have been very well anticipated that this is going to happen, after all they must have all the stats from the previous HacktoberFests.

We need to understand that Hacktoberfest is an marketing event. And all these spam and blogs posts around it is definitely serving their main purpose, there is no such thing as bad publicity indeed. Of course I could be totally wrong, all of this could be completely unintentional.

Now, coming to the Indian side of the ongoing issue. Out of all the countries why does India have the most spammy PRs ? As a seasoned armchair philosopher, the following are my thoughts on the “Why” of all this.

The Signalling Problem

Whenever an online article/documentary/report claim to give a view into the “real” India, most of us (Indians) get offended.

I was in the bubble which was in the “offended” bucket for long time. The reason was simple: I, the people around me and the societal structures around me were all part of the same bubble. We do not have to worry about paying school fees or rent, we do not have worry about sending part of salary back home just so that our parents can buy groceries to survive..

I can go on and on, the overall gist is that I got offended in seeing what other people claim as “real” India, because that India was not part of my reality. Sure, I saw that when travelling from one place to another, saw that on news, read about that in the newspaper; but I was at a very safe distance from all of that.

These days, I try to not get offended when people say India has problem X or problem Y, because I realise I am living inside a cocoon where all of my needs are met, and one of the most serious issues my country is facing right now is witch-hunting nabbing the “drug mafia” amongst the Bollywood celebrities who allegedly murdered an actor by giving CBD oil (The absurdity of all this is unfathomable)

I mention all of this, because whenever I read posts which show India in some bad light, there is always someone somewhere who gives anecdotes of how that is not true.

Here are some facts: we have a culture of extreme signalling. Signalling is the core building block of our society, most people don’t even realise that how big we are into signalling until they study about “signalling” as a phenomena and start becoming conscious of it. You notice that in the way your parents view you, how you make choices, how people around you make choices.

Of course, people will say “that is not an Indian problem, signalling is just a social construct and literally everyone does that on some level”. I am not denying not that, the very fact that I am writing this post is a kind of signalling I am doing.

It is well known fact that India is one of the densest countries on our planet, extreme siginalling is just one of the consequences of the myriad social problems created by high population density. Signalling in an high density environment is now ever more important as the people you interact or the people who notice your activities/accomplishments are multiples higher than in any other place on the planet.

We people like the wear the “tightly knit society” as badge of honour, let me tell you this “tightly knit society” has done more harm than good. We Indian people have no India of the amount of mental harassment we all go through, because that is just normalized as being just phases of “life”.

The prevalence of extreme signalling brews the classic and infamous herd mentality in our minds. In middle school children have dreams to become a Pilot in the airforce, or maybe a Police officer, or maybe a Opera Singer ! But by the time of high school, everyone is just either on road to become an Engineer, a Doctor, a Lawyer, a Chartered Accountant …. or a failure. This may seem harsh but that’s how most of the society operates here in India.

Now once your track is chosen for you (or fortunately, if you get to choose the track), you have some set goals you “must” achieve because they guarantee monetary success with an extremely high probability. If you are Engineer, you need to get into IIT. If you are a Doctor, you need to get into AIIMS. If you are doing management, you need to get into IIM.

Now, if you are an Engineer then your success criteria is not just any IIT, it should be one of the top ones (Bombay, Delhi, Kharagpur, Roorkee) and not just any branch in IIT, it should be Computer Science. Because that’s how you get Rs 1.5 crore “packages” (equivalent to 200K USD).

You can give me examples of how not all IIT toppers choose “Computer Science”, but that’s not the point, for the vast majority of people (more than a million), the success criteria is: getting Computer Science at IIT Bombay.

As might know (or have guessed) getting a seat in IIT Bombay is next to impossible for more than 99% people giving IIT, so they “lower” their goal by aiming for other IITs, but “Computer Science” remains the top priority.

The Jugaad mentality

Hacking systems is so ingrained in our society that we have a word for it: Jugaad. The whole scene with Hacktoberfest is just a demo of our Jugaad skills. Wait till you find out that most of the research papers published in India are fake. Even the orthodontist I was visiting for my checkup turned out to be fake and had forged her certificates (as told by by one other dentist).

The coaching industry actively promotes “cracking” these exams, and I am so tired writing on that topic that I do not want to write more about it here. Refer to my post on this here. Some children start joining these coaching classes from as low as 5th grade! (apart from doing regular school), just to be able to “crack” the IITs after 7-8 years !

Now how is coaching industry a Jugaad ? Simply because from the point of view of IITs, attending regular school should be enough to prep you for the exams (I think). These coaching industries have made the process significantly harder as you can’t even hope of getting a low tier IIT without attending the coaching classes.

The IIT coaching industry is minting fat cheques out of this entire situation, look at these ads on the front page of the news paper. These ads create a vicious cycle by reinforcing the core “signalling” construct of our society.

Do not quote me on this, but I think the IIT coaching industries make more money than the IITs themselves. I understand the purpose of IITs is not to directly generate money, but if it would have more resources they can probably improve the infrastructure, employ better professors and improve the overall level of education.

Computer Science Education in India

For vast majority of engineers in India, Computer Science is one of the subjects you study in order to succeed in life. Just like you study Chemistry, you “study” Computer Science and once you learn all the “concepts”, you get a good job. (Apologies for so many quoted words, I can’t help putting them in quotes because they carry so much weight for me).

When people are in an average Indian college and if they are are lucky, information about some permutation/combination of the following is spread amongst the students: Codechef, ICPC, DS/Algorithms, Infosys -> road to USA, GSOC, Github, Google, Facebook, 1.5cr “package”, MBA, MS in USA, .. yada yada yada.

Of course this is not given by college authorities as they have no idea of the external world.

Us Indians are very good at following rules (or guidelines depending on the way you look at it), just tell us what to do, and we’ll do exactly that. We do not need any reasons or justifications for the most part, (we haven’t been taught that by the society). Just follow the rules and everything will be “set” for you is what everyone says and does. See a sample of how we learn to write emails and prepare for software interviews

But why for Hacktoberfest?

Coming back to Hactkoberfest, it fell prey to the Jugaad nature of our existence. I personally know people who in-spite of being well paid software developers created dummy repos to create a circle of PR contributions for past Hacktoberfest events.

Of course, wherever there is any sort of incentive related to some metric, people will try to game that, whether that is LinkedIn connection counts, number of likes on Facebook or number of impressions on Instagram. There is nothing specifically Indian about it.

But there is one thing Indian that, the sheer number of people here who are out there to game everything which can give them some “profit”. The % of these people vary by their personal financial factors, perceived financial factors and of course the plethora of social constructs around them.

The problem is for most people gaming the system, that’s the actual system for them. They see hacking the system as part of the entire system itself. Things get so convoluted due to limited sources of information that people automatically think that hacking is just part of the process and something which is expected out of them. It’s a messy coordination problem that we have caught ourselves into.

The way of the jugaad is now applied to things unknowingly by people even when it is not really required. So far everything we have done needs some sort of Jugaad, so why would we not do that for getting t-shirts, getting into colleges, getting jobs, getting promotions, getting salary increments ?

After all the entire system is corrupt, right ? Everyone is corrupt, So why should I be the naive one and NOT try to actively game the system ? DigitalOcean is a big company, what would they lose if I just get one T-Shirt ?

^ That’s the thought process of a typical Indian developer.

The problem with high population density

Everything mentioned so far can be in some sense applied to humans all around the world, but the amount of people in India makes things more complicated.

It complicates things because the society now has formalise / organise all this knowledge and information, which then slowly seeps itself into our collective unconscious. There is no overseer of this process, parents single-handedly pluck information from the collective unconscious and feed that to their children. And it’s NOT their fault, they are doing their best. If they knew what exactly to do, they wouldn’t have ended up in a dire situation in the first place, so of course they cannot rely on their own conscience, which I think is absolutely the right thing to do.

Once these ruleset(/guidelines) are plucked from the collective unconscious and are concretised by the news, media and society, then an individual cannot question them. Because why would you question them ? Millions of others are doing the same ? Are you claiming you are better than them by questioning the rules ? How will your parents get bread if you don’t study hard enough and get a job in the city ?

I want everyone reading this to understand that you have a privilege that, sorry to say, has blinded you about the real issues and problems we are facing. Most of the humans of our planet and by that, most of the humans living in India, are doing nothing wrong (from their own frame of reference), who am I or you to say what is Right or Wrong… for them? The situation gets so complex and intertwined that what is Right or Wrong for the other person takes the backseat, all that matters is what is Right or Wrong for me.

Hierarchy of Needs

Armed with the wisdom from the unconscious all of us are ready, with just one mission: financial success. Trust me, when I say this, “Financial success” is what matters the most for us Indians. When I see someone who decides to push back “financial success” in order for other priorities, I have massive respect for them.

You need money to buy stuff, your family needs money to buy stuff; after all lack of money is our main source of unhappiness, right ? So keep your head down and just work towards that.

Trust me, I cannot begin to be thankful of the privilege I have in my life. My sole purpose of life is not just to be “financially successful” and provide resources for people around me. If I was part of the real India with the same name, intelligence, whatever, I would sure as hell be in a completely different situation where the only thing I would care about was earning money in order to satisfy the basic needs of me and people dependent on me.

When I say basic needs different thoughts can rise up, for some basic need is having just a house, for some basic need is a 3 bedroom house because they have their parents or grandparents to take care of, for some basic need is getting a job so they can get married and have children, for some it could be to just buy a car. In all, it is pretty subjective and you cannot convince people to let go of their basic needs, because for them, it is their basic need.

For the average software engineer aspirant in India, the basic need is to just get any 9-5 job. They don’t have the privilege to choose which company they want to work for, so they will join the first company which offers them a job.

Closing Thoughts

It is without doubt that India has produced a lot of good programming talent and software, but we should not ignore the elephant in the room: most engineering graduates in India are not fit for hiring

People everywhere on this planet are just slaves of their circumstances and the societal constructs which surround them. When push comes to shove (which is the permanent state for most Indians), we only care about one thing, survival.

So, it doesn’t really matter how one gets a Hacktoberfest T-Shirt, as people don’t really have the conscience to see the consequences of doing that. As everyone around them is applying some Jugaad, so why not join the herd ?

Probable Future

• The probable future is simple to predict, the things which are happening today will keep on happening. This is a struggle for basic survival, and what others think takes the backseat in this game.
• Coding interviews will become harder. As I mentioned in my post, coaching industries will pop-up to help “crack” coding interviews, GSOC and maybe later HN posts. Trust me it is completely possible, all it takes is one Indian Google Engineer to create a YouTube video and say that he had X karma points on HN mentioned on his resume and that’s how his resume got shortlisted for Google.
• Big coaching industries in India have already started charging Rs.1500 for lessons on C++ STL. Without doubt, this will continue to happen on a larger and broader spectrum.
• YouTubers will continue to provide low quality tips and tricks for people.

  • See what is happening to GSOC (I searched “GSOC” from a machine in Europe, to prevent geo based search results):

  • Even Google’s Foobar challenge is not spared:

• People will get offended on how I am denigrating Mother India. These people will questions my credentials and also point out that I am not a thought leader of the industry therefore all of my views are wrong by default.

Improbable Future

• Somehow people will realise that the metric they should focus on is the love for computers, respect for other developers and the social impact of programming on the world; not ranking in some programming contest, or number of Github contributors, or number of Github stars etc.
• Somehow good Indian software engineers will create quality content, blog posts and show what their actual job are like. They need to show that cutting edge software engineering happens not just in FAANG, but also in startups.
• Children in school have complete visibility and guidance on what lies ahead. Parents will stop projecting their own failures/insecurities in life on their children.
• Somehow Indian parents will become self sufficient, so that children are free to do and discover what they actually want.
• Failure is appreciated in society.

Find discussion on HackerNews here.

This essay is in continuation of Source [0]

We all are vaguely familiar with the concepts of bubbles. Bubbles in the sense that each one of us is present in them and that bubble dictates which kind of information you consume, love, hate, and such. Bubbles can also be called a belief system. A belief system is not just about whether you believe in god or not, but it just tells in a broad sense what one believes about everything.

As we grow up, we metaphorically also keep growing our bubble by having new experiences, meeting new people, going to new places, discovering new knowledge to state a few.

Bubbles are generally used in a negative sense, for example, “That person seems to live in a bubble, he does not care about politics”, but bubbles are very helpful. You see, we are unconsciously aware of this fact, you may not call it a bubble or not even call it anything, but you know everyone (including you) has a bubble. And it’s just not people, we also put non-living things in a bubble. Like a religious book almost always gets put in a dogmatic bubble.

By putting things and people inside their own bubble, our mind now takes the call as to whether it makes sense to read/question/argue/understand something/someone or not. It does this decision making by seeing your own bubble and the other thing’s bubble, if they both align, i.e. they share some common beliefs, then the mind can allow us to continue, otherwise, the mind thinks that doing the activity in question will be a waste of time and energy.

There are many positive benefits of doing this, that’s how we have diverse thoughts, things, and persons. If the mind does not do it, countless amounts of energy will be spent by people in doing things that they don’t really require in some sense.

Conventional schooling systems do the same thing, they try to stop this automatic decision-making process of mind and forces us in doing things which don’t really matter in the long run. You are not allowed to question why you are studying a particular subject. A counterargument to this can be that during childhood our bubbles are not mature enough, hence the way the schooling system is designed. My counterargument to that is, in retrospect, I do not think it managed to change my bubble significantly or increase the rate of how my bubble grew, at best it created a false impression for me (and the education system) that it grew bigger by passing exams which test regurgitation abilities.

We have seen the positives of how the prioritization process of aligning bubbles done by our brain, that is it helps us in saving time and energy and helping us to grow, now it is time to see some downsides of that process.

One important point to note that this decision process (of choosing which bubbles are good/bad for you) is not always a conscious process, it can be seen a long-running process in the subconscious mind which sometimes comes into the conscious realm. Whenever you think you make a decision, it’s a conscious process, in other cases, your mind is making a lot of decisions subconsciously.

Let’s see an example, consider a person who is not interested in science and related topics as they consider these things to be “hard”. That person now sees a video on YouTube titled: “Deep Dive into our Universe”, chances are very low that a person will attempt to watch that video, as the decision would unconsciously have been made by the mind. There is a caveat, due to the way modern internet operates, they will probably not even get into this situation as the recommendation algorithms will rarely show this content to the person. The internet giants have a better understanding of what your bubble is (better than we know it ourselves), therefore it will show you content that aligns with your bubble so that you consume it and then they can make money off ads. I am not saying that I have a better alternative, just stating the facts so you know how they operate by representing our bubbles via machine learning systems, using extensive monitoring/tracking.

Consider that this person had the option to watch the video, but they did not, reasons can be that they thought that they would not be able to understand, why to spend time in it, how does this help me, and similar thoughts. Here’s a fact, they might be completely right, watching this video may really not serve any purpose to them. In this way, bubble based judgments are really helpful in most cases.

But let us consider another possibility, that they did watch the video, and it turns out that the video was deeply instructional in nature. They thought they would not understand any of it, but after watching some part of it they understood 20% of the content. That’s a great thing!

So, there is always a possibility that something that our mind clearly ignores or does not give much more thought to, it is possible that there is more meaning present there. To look at the extra meaning, you need to put in a conscious effort to discover or extract it.

“Effort” is an interesting word, from the following which thing takes more effort? watching a 1-hour movie, or 1-hour video of a college lecture. Again, if the lecture is in a subject you are interested in (i.e. aligns with your bubble) you may need almost zero effort to watch the lecture, but as you may know, it takes considerable effort to watch the lecture video if it does not align with your bubble, and most probably you won’t watch it entirely, unlike in an actual lecture where you would have to sit through it even after you decide to stop making the effort to understand it, just to keep your physical self present in the lecture hall.

Coming back to the judgment aspect of it, I want to give another personal example. Sometime back I wrote about a particular company charging too much for coding related courses, one of the comments from a major stakeholder in that company said: “You are making these claims as if you are intellectual thought leader of the country”. Obviously he was projecting what kind of bubble I need to belong to make sense to him, instead of looking at what I said and countering that.

I see this is a major problem that limits all of us to new ideas, every time I have a judgment about something, I try to think, is that because of what someone said, or is that because who said it, or is it a mixture of both cases. I saw an interesting comment on YouTube on a video, it said: “A debate should not be about who is right, but what is right”.

Now, let us resume from where we left in Source [0]. For the sake of clarity, I am going to reiterate our established common ground.

1. Philosophy is not just about talking in the air, it can contain some meaning which can in turn help us in widening our perspectives.
2. Most religions have an essence that is deeply philosophical.
3. In the case of India, a large number of sages spoke of deep philosophical concepts.
4. We have a set of belief systems or bubbles in place which help in consuming things that align with who we are, but on the other hand, it can also prevent us from discovering something new.

If you have reached so far, I hope that this content is aligning with you, so the purpose of writing this is fulfilled, I hope you continue reading but as a cautionary warning, I want to mention the target audience of this particular series of blogs for the future posts.

Without digressing too much, I want to point out that there are several views on the meaning of life, one view says that the meaning of life is to do good things, another says that the meaning of life is to achieve one’s true potential. The one view which I relate to a lot is the view that meaning of life is to gather knowledge and wisdom.

By saying that, I am not claiming that other views are in some sense of lower stature than the view of knowledge. I just state that so that you can see which bubble I belong to (and in turn this series of blogs), this will help you in judging whether you want to spend more time reading it or not.

With this in mind, I just want to point out that I have often found that expanding our own bubble never hurts and something good often comes out of it. For instance, reading the Bible would not magically convert you into a Christian, you’ll just create your own interpretation of what it is written in it, which will definitely give you a fresh perspective on the world i.e. expand your bubble.

In the next blog post, I will attempt to introduce you to some concepts related to the study of the nature of god and consciousness in ancient Indian philosophy, hopefully, the bubbles align and help in expanding yours.

Series of essays exploring Indian Philosophy/Sprituality

I’ll start with a word: Dissatisfaction. It is a word most of us try to run away from. We do not want to feel dissatisfied. Why do we not want to feel dissatisfied, but we are okay with feeling satisfied? Is life full of satisfaction with small pieces of dissatisfaction in between (like raisins on a cookie), or the other way round, wherein life is full of dissatisfaction inside which we find small moments of satisfaction. Or the extremes like, life is either just dissatisfaction or just satisfaction. These statements are pretty obvious but can be inspected a bit more (As later you will see this process of continuous inspection leads us to some uncharted territories).

These are important questions. The importance of something is of course subjective, so let’s assume them to be important for the sake of reading this text.

A possible reaction for a layperson while reading so far maybe: this guy (me, writer) is depressed about life, or some thought close to this in a similar direction. The reason why I say so, is because I would (could?) react in a similar manner. Whenever someone writes about something so-called “sad”, the knee jerk reaction is, “this person must’ve been through something difficult”. It is an important point to consider.

Another possible reaction is: Why is all this important? What is your point? Why are you making pointless statements/questions? Again, these are possible reactions, but just for the sake of reading this text, let’s assume that you may find something useful here.

Now that we’ve loaded a big pile of assumptions, I’ll come to the central point of this text.

Let’s look at some facts, most of our modern education/upbringing is orchestrated in a way that places importance on our external existence. And this has brought us tremendous success/results. Human life expectancy has drastically increased in the past 200 years, and of course, with the advent of the internet, we have started to innovate more rapidly.

Now the age-old question, have we become happier? Since this question can be cringe-inducing, let’s look at it from a different perspective.

From the human evolutionary standpoint, the modern world which we see ourselves in has been around for a minute fraction of the entire human existence (Homo Sapiens have been around for about 500,000 years).

And we have not grown any smarter, in the sense that our brain has not evolved in the past 50,000 years or so. Of course, here my assumption is that our smartness is proportional to the size of our brain.

This fact raises a question: if we assume that we have the same intellectual capacity for the past 50,000 years, why did modern science only emerge recently, i.e. in the past 2000 years, why did all this not happen any earlier? Why did we discover nuclear power now and not 30,000 years ago?

Another way to look is, what were the people before our modern age doing with their life (when they had the same intellectual capacity).

These are treacherous grounds I am entering here. You may notice that I did not use the term IQ, instead, I went ahead with something like “intellectual capacity” in order to comment quantifiably on our ancestor’s intelligence. The thing to note here is that IQ (or any other measure of intelligence) is a modern way to measure modern intelligence, so it can not be applied to them (humans thousands of years ago).

To give an example, the human species survived and evolved for millions of years all without the presence of spoken language, is that not sufficient display of intelligence? A million years is an unfathomable duration of time.

It is difficult to comment (from my uneducated perspective) what were the intellectual pursuits of our ancestors. We definitely see the beginnings of religions/cults at that time, as a way to organize and create a system of information distribution glued together by doctrines and dogmas.

What I can comment on is what people in ancient India did.

Before that let’s take a small detour to understand philosophy, to establish some common ground.

Philosophy is often misunderstood as “talking in the air” or air talk. One way I think about philosophy is that any questions/statements that cannot be answered literally or create a feeling of profoundness or create a deeper need for further inspection, can be considered as philosophical. You can obviously find better definitions in other resources, I am not a philosopher.

With our modern age, India’s philosophical history is tied very deeply with religion, even ingrained in it. In fact, it’s not just about India, most of the modern religions are deeply philosophical in essence.

Coming back to ancient India, you may already know that India was a country of sages, who were deeply revered. I do not feel hesitation in saying that these sages are what can be called philosophers in modern-day. But since they wear saffron and pray to some god, their statements/teachings are consumed very rarely by non-religious people, I will go one step further and say that these teachings are not even understood clearly by most religious people.

Which brings us to the questions, why should we even listen to these great ancient philosophers? What “knowledge” can they have which can be applied to our modern world, which is so drastically different from ancient times? How can these philosophers say something which can apply to me?

I believe these questions are completely valid, and I will talk about them later in further posts.

References

• Evolution of Modern Human Brain Shape
• Wikipedia: Indian Philosophy
• Wikipedia: Life Expectancy

Next post in continuation of this: Source [1] - Bubbles

A look at into how I used Vim to notes about frequently used shell commands at my new job

Before I joined my job, I had been a heavy Windows user. Apart from the “UNIX Programming” course in college, I had barely used any of the shell commands.

In the initial days of the job, in order to run a previously executed shell command, I used to press the up arrow key on the keyboard to search for it.

Shell Reverse Search

Turns out there is a neat way to search through your previously executed shell commands (All executed commands are stored in the bash history, for zsh it is the ~/.zsh_history file). All you have to do is press CTRL+r and then type some parts of the commands that you can recall.

For instance, if I know that sometime back, I had used a command to figure out the role of a particular Redis instance. Then all I have to do is <ctrl-r>role and the desired command while appear automagically !! (Keep pressing <ctrl-r> to cycle through the results)

Aliases

One day, I was looking at my co-worker’s laptop while he was debugging some issue, and he started typing mysterious commands in the terminal like gco master. Turns out zsh comes with a set of aliases, which are like shortcuts to entire shell commands.gco is an alias for git checkout. If you feel that there is a command that you need frequently, it is a good idea to make an alias for it. This avoids typing the entire command, and you can always reverse search for the alias, as it should be easy to remember.

To see existing aliases, you can run alias -L. Creating a new alias is as simple as adding

alias ping_redis='redis-cli -h 100.100.100.100 ping'

to your rc file (For zsh it is primarily ~/.zshrc).

As you keep using the terminal, it may happen that you need some commands which are infrequently used but are important. For example one time configuration commands, installation steps, or some important queries to your database.

Making an alias does not make sense for each of these commands as they are not frequently used and you’ll get overloaded with aliases to remember. Reverse search might not work if you don’t remember any part of the command (or the bash history file got deleted/truncated).

I often found out that I had important commands scattered in different documents. Some of them might be in the OS X Notes app, some in an always running Sublime Text window or some found out by grepping ~/.zsh_history.

Once, I would’ve found the command; the next step was to copy, go to the terminal window, paste and execute that command.

Vim

Why not use Vim?

For me, I keep a commands.sh file to take notes regarding all the important commands and use a handy little setting in the vimrc file to execute those commands from the file itself. Check out the demo…

You can select the appropriate shell command in visual mode and run them by pressing r

Here are the appropriate vimrc settings for zsh. I had to do some digging on StackOverflow to get it to load my aliases.

set shell=zsh
vnoremap r :'<,'>w !zsh

You need to ensure that your aliases are in .zshenv, or you can make .zshenv a symlink to .zshrc.

Keeping the shell commands in commands.sh will allow you to write commands that you can later automate. For instance, if you repeatedly SSH into a machine to run something in the Rails console, you can keep the following in your commands.sh file instead:

ssh -t 100.100.100.100 "cd /some/dir/rails/;bundle exec rails console staging <<< 'Trash.delete_all'"

What are your thoughts? How do you manage the growing set of commands you work with?

A neat way I found out today was to use comments to tag shell commands. For example:

find . -name "*.sh" #find shell scripts

Once you execute the above, you can reverse search by doing find shell scripts. Sweet!

Helpful Resources:

• https://stackoverflow.com/questions/2575545/vim-pipe-selected-text-to-shell-cmd-and-receive-output-on-vim-info-command-line/5373376#5373376
• https://unix.stackexchange.com/questions/113462/how-can-i-make-shell-aliases-available-when-shelling-out-from-vim/197135#197135
• https://unix.stackexchange.com/questions/26245/how-to-quickly-store-and-access-often-used-commands/26263#26263

After reading numerous posts online for using Jekyll and Github Pages together, I faced couple of issues. Lets take a look and see how I addressed some of them.

• Jekyll 3 now recommends using Gem-based themes. I was not able to find themes other than the ones mentioned here.
• The ones supported by Github are not exactly blog like.

After spending considerable amount of time looking for a perfect theme (And giving up numerous times). I decided to go with the whiteglass theme.

I cloned the repo and had the blog running locally, it was time to push to my .github.io repo. After, refreshing 3-4 times all I was staring at was a blank white screen. Soon, I got a mail from Github regarding a build warning.

You are attempting to use a Jekyll theme, “jekyll-whiteglass”, which is not supported by GitHub Pages

The Fix

Instead of serving a Jekyll Project directly from the master branch of the github.io repository, we can instead serve the static files generated from the jekyll build command, i.e. the contents of the _site directory. This way, we will remove dependency on Github Pages to build the Jekyll Project, instead it will only serve the static pages.

Since, the master branch will have the “built” website, where should we store the actual Jekyll Project ?

In my case I have stored it in the development branch inside the same repository. I will work on the blog in this branch and will push the built website to the master branch. I have seen some repositories where people have written Travis jobs to do this exact thing. I feel that is an overkill, a slow process and downright exploitation of a free service. Please correct me if I am wrong.

I have written a small shell script which will build the website and push the changes to the master branch. Here it is:

#! /bin/sh

BUILD_DIR="./_site"
TMP_DIR="/tmp/jekyll_build"
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)

# Delete old build files if any
rm -rf $BUILD_DIR

# Build
bundle exec jekyll build

# If Build succeeds
if [ -d "$BUILD_DIR" ]; then
    echo "JEKYLL: BUILD SUCCESS"
    rm -r $TMP_DIR

    # Move BUILD_DIR to /tmp
    mv $BUILD_DIR $TMP_DIR

    # Change branch to master
    if git checkout master; then
      echo "SUCCESSFULLY SWITCHED TO MASTER"

      # Cleanup
      ls | xargs rm -r

      # Get the build
      mv $TMP_DIR/* .

      # Commit
      git add .
      git commit -m "Deploy $(date)"
      if git push origin master; then
        echo "DEPLOYED SUCCESSFULLY !"
        echo $(basename $(git remote show -n origin | grep Fetch | cut -d: -f2-))
      else
        echo "DEPLOY FAILED !"
      fi
    else
      echo "CANNOT SWITCH BRANCH, ABORTING"
    fi

else
    echo "JEKYLL: BUILD FAILURE"
fi

git checkout $CURRENT_BRANCH

This script lives in the development branch.

After I am done doing my changes to the blog. I push them to the development branch, and then do ./deploy. The script will then make a new commit on master having your latest masterpiece.

Why I don’t like the gem based themes.

I have added the deploy script as a post-commit hook, but unfortunately some changes do not require a commit.

This is because most of the theme related code is now inside the gem’s directory (i.e. not inside my git repo). So, say If I want to change the Google Analytics tracking script, then I’ll need to do the changes in the google_analytics.html file. Since, this file is inside the gem’s directory (in my case at /Users/pulkit/.rvm/gems/ruby-2.2.5/gems/jekyll-whiteglass-1.3.0/_includes/), I cannot make a commit on development to reflect this change.

Gem based themes are good for people who are new to Jekyll and do not wish to modify the themes they use. Since, most of the theme related code is now present in some other directory, users will not be overwhelmed from the directory structure when they do jekyll new. But sooner or later, you’ll need to make some changes to the theme (changing the font size for example). Doing those changes in some other directory and not inside version control is pretty cumbersome and will hurt in the future.

UPDATE: Found out this https://jekyllrb.com/docs/themes/#converting-gem-based-themes-to-regular-themes, this will make your git repo to have all the code ! But then, what is the point of gem-based themes ?

Discussion on HackerNews.