The concept of geography/location agnostic enterprise is very simple. Whether I am in Beijing or Boston, Kent or Kentucky, New Delhi or New York, Silicon Valley or Silicon Wadi, Qatar or Quebec; I should have access to a standard set of tools to run the company. Moving around geographic locations are hard challenge for any enterprise. One of the first problems we wanted to solve is how quickly we can deploy some of our tools to countries around the world. Part of the reason why I envision my mission with nanøveda to be a geography/location agnostic enterprise is because the problem we are trying to solve is universal. We want people around the world to have uniform access to our technology. It will also help us become better in business too.
To solve this problem I am in search for answers. I found a brilliant solution a few days back. Recently, I got an invite for a trial of Google Cloud Platform (GCP). Google was kind enough to give me a $300 credit towards creating applications in their cloud platform. I was very excited to try this cloud computing platform from one of the leaders in computing. Finally, last Friday, I decided to explore GCP. For me cloud computing brings two advantages: 1) Zero time and cost overhead of maintaining a set of in-house linux servers; 2) Creating a truly geographic agnostic business. I am running an Ubuntu 16.10 workstation for running machine learning experiments. Most of the tasks handled by this workstation has already started to overwhelm its intended design purpose. Therefore, I was actively looking for solutions to expand the total compute infrastructure available for me. It was right at this moment, I turned to Google to help solve our compute infrastructure problem.
I have never used GCP before. Therefore, I had to go through a phase of experimentation and learning, which took approximately a day. Once I learned how to create a virtual machine, everything started to fall in place. To check if the vm is seen properly by the guest os, I ran some basic diagnostic tests.
cat /proc/cpuinfo cat /proc/meminfo
The GCP has an interesting flat namespace Object storage. The feature is called Buckets. The buckets feature allow the virtual machine to share data from a remote computer, very conveniently over a web interface. Google has a command-line tool called gsutil, to help its users streamline the management of their cloud environment. One of the first commands I learned was to transfer files from my local computer to the Object storage space. Here is an example:
gsutil cp /home/rahul/tf_features gs://module-name/tf_features/
Once I learned file transfer, the next step was to learn how to host a virtual machine. After I set-up an Ubuntu 16.10 virtual machine in the cloud, I needed to configure it properly. Installing necessary linux packages and python libraries were easy and fast.
sudo apt-get install unzip sudo apt-get install python-scipy sudo apt-get install python-pip sudo pip install keras sudo pip install tensorflow sudo pip install pandas
After the vm was configured to run the code I wrote, the next step was to test file transfer to the vm itself. Since vm and the Object storage are beautifully integrated, file transfer was super quick and convenient.
gsutil cp gs://module-name/tf_features/*.zip /home/rahul/ gsutil cp gs://module-name/tf_features/*.csv /home/rahul/
After the code and all the dependent files were inside the cloud vm, the next step was to test the code in python. The shell command below executed the code in the file test.py.
python test.py
Since some of the code execution takes hours to complete, I needed a way to create a persisting ssh connection. Google offers a web-browser based ssh client. The browser ssh client is simple, basic way of accessing a virtual machine. But, for longer sessions: creating a persistent session is ideal. Since my goal is to make most of the computing tasks as geography agnostic as possible, I found a great tool for linux called screen. Installing screen was very straightforward.
sudo apt-get update sudo apt-get upgrade sudo apt-get install screen
Once screen was installed, I created a screen session by typing screen in terminal. The screen session works like the terminal. But, if you are using ssh, it allows persistence of the commands being executed in the terminal even after the ssh terminal is disconnected. To quit screen just use the keyboard short cuts: ctrl+a followed by ctrl+d.
To resume a screen session, just type screen -r in vm terminal. If there are multiple screen sessions running, then one will have to specify the specific session that needs to be restored.
screen -r
The ssh + screen option is a life saver for tasks that require routine management, and needs a lot of time to execute. It allows a vm administrator to convert any network connection into a highly persistent ssh connection.
The combination of Google cloud Object storage, ease of networking with the vm, ssh and screen has allowed me to transfer some of the complex computing tasks to the cloud in less than a day.The simplicity and lack of cognitive friction of the GCP has taken me by surprise. The platform is extremely powerful, sophisticated and yet; very easy to use. I have future updates planned for the progress and evolution of my usage of GCP for our start-ups computing needs.
I am still amazed by how easy it was for me to take one of the most important steps in creating a truly geography/location agnostic enterprise with the help of Google Cloud Platform. I have to thank the amazing engineering team at Google for this brilliant and intuitive cloud computing solution.
This work is done as part of our startup project nanøveda. For continuing nanøveda’s wonderful work, we are running a crowdfunding campaign using gofundme’s awesome platform. Donation or not, please share our crowdfunding campaign and support our cause.
Donate here: gofundme page for nanøveda.
1 comment