[TOC]
Although in popular parlance “programming” means writing code on a computer, to a computer scientist, programming is a mathematical activity, with or without a computer. In the recent past, most programmers were mathematically trained computer scientists who also wrote code. But today, the majority of coders in the IT industry are not computer scientists; they write code to automate business processes, relying exclusively on coding skills and without resort to mathematics. They call themselves developers.
Developers are focused on the difficult problem of implementing efficient, maintainable, business applications that run modern society and world economy. Computer scientists, too, are concerned with such important, practical matters, but, additionally, they are keen on broader, deeper, theoretical issues like accuracy, precision, efficiency, computability, and proof. However, the IT industry in general is intolerant of such theoretical matters, because for several decades, line-of-business systems have traditionally been built with exclusive reliance on business operations knowledge and system implementation skills. From the industry’s pragmatic stance, if a system works in practice and is making money, there is no need for theoretical justifications.
The threshold question a young person must ask himself, thus, is whether if he is interested in making a living as a developer or if he is keen on pursuing a career as a computer scientist. What is the difference between the two? By way of analogy, a developer is to a computer scientist a carpenter is to a civil engineer. Like all analogies, this, too, is imperfect, but it does serve to highlight the differences.
He who wishes to learn to build chicken coops needs only take up carpentry. Although carpentry is an ancient craft with deep traditions and immense social import, there is no carpentry university programme; the apprentice may immediately begin toiling with tools and materials of his trade, picking up necessary new skills on the job. But one who aspires to design and build slightly larger structures—like skyscrapers and bridges—must study civil engineering for several years, with pencil and paper as his primary tool and material. A carpenter needs to know only woodworking. But an engineer must know properties of myriad construction materials, advanced mathematics, structural design, fluid dynamics, process management, supply chain management, project management, personnel management, and government regulations. In other words, a carpenter must know the advanced practical skills, but the engineer must possess broad and deep theoretical knowledge, in addition to the basic practical skills.
Similarly, a developer is expected to know all the pernicious quirks of the programming language he uses and is expected to acquaint himself with the peculiarities of the application programming interface (API) he uses. But even a highly experienced developer will not be tasked to design a new, original neural networks that is faster and more accurate than all the existing ones. On the other hand, a computer scientist specialising in machine learning is expected to design a new, more accurate algorithm, prove its correctness and efficiency, publish a peer-reviewed scientific paper, and implement a prototype, but he is not expected to know how to write the most efficient production-quality code on a particular embedded microcontroller board. But many computer scientists additionally perform such practical tasks, alongside their theoretical duties.
The reality, however, does not always match expectations. Although a newly minted civil engineer will never be required to apprentice under a carpenter, most young CS graduates must enter the IT industry as junior developers. Today, only a few unusually talented ones go on to design large, complex, life-critical systems; the majority are never given the opportunity to employ their CS knowledge. That is, they live out their technical careers as developers and subsequently transition into management. But they may get the second chance, when, and if, they ascend to technical executive positions—CIO, CTO, or VP. In these key positions, one gets to use his CS knowledge in directing the development efforts, albeit from afar without being hands-on, while shouldering corporate and financial responsibilities.
Top graduates from the few top universities tend to remain in academia. But many among those who stayed in academia find it hard to resist the call of the market bell; after a few years, they end up trading their feeble remunerations of academia for the fairer prospects of industry.
Usually, high-achieving CS undergraduates go on to graduate school, where many advanced, theoretical courses are taught. But some CS graduate students are from other STEM fields, and some even from liberal arts. A few CS graduate students are part-timers who work full-time in the IT industry. Most graduate students are sponsored by the university, and they work either as teaching assistants (TAs) or as research assistants (RAs). TAs teach introductory undergraduate courses as part-time instructors, which is a great way to gain teaching experience. RAs work on professors’ funded research projects, designing algorithms, implementing them, and publishing scientific papers on various subjects. This is how future CS researchers are trained.
After earning their PhD, the graduates go off to some other university and become associate (junior) professors. Others join research programmes elsewhere, as post-doctorates. These researchers are typically sponsored by the government or major IT corporations. It is an unwritten rule in academia that a graduate must leave his alma mater and join a different group with a different intellectual tradition, thus diversifying and strengthening numerous academic bloodlines.
Everyone in academia is a teacher; even those with strong preference for research have to teach. In addition to teaching, they must publish peer-reviewed scientific papers, regularly. A teacher reaches the pinnacle of his career, when he is appointed a full professor at a university. The other path to a professorship is to do great work in the industry at companies with strong academic cultures, then join academia in later career. Many a Bell Labs alumnus has taken this route.
The majority of CS graduates enter the IT industry, and the large portion thereof become line-of-business developers. Only a small percentage of them work in specialisations that require them to employ their hard-earned CS knowledge. Even though developers do not dabble in fancy theoretical concepts, they perhaps have more fun building billion-user social media systems, and they are compensated much better than their counterparts in academia. So, a CS graduate working as a developer has rewards all its own.
Until recently, system maintenance was a major cost of IT operations. Armies of system administrators were needed round the clock to keep a large data centre in good order. The administrators are constantly busy, monitoring services, maintaining aging hardware, and repairing damaged components. With the advent of cloud services, self-service provisioning, and automated monitoring, most private- and public-sector organisations are now able to avoid hardware ownership and its incidental costs: data centre, backup site, administration staff, training, preparedness, etc. The work that system administrators used to perform have now been either automated away or are subsumed under software development teams. The primary employers of system administrators today are large, cloud service providers, and due to the growth in this area, system administrators continue to be in demand. Note that although some CS graduates work as system administrators, a CS degree is not necessary to become a competent system administrator. There are many training and certification courses for system administration.
Another high-demand role is line-of-business coder, the job the public thinks everyone in IT holds. Starting around 2010, JavaScript and its Node runtime environment took the business software sector by storm. JavaScript, despite its well-known faults, is a simple language that novices can learn quickly, and is an effective language in the hands of an experienced coder. The Node ecosystem has ready-made tools for every imagined need, most are trivial, but some are vital. And more importantly, JavaScript applications run on mobile devices, web browsers, desktops, servers, and embedded systems. Companies can now have one development team and one code base for all these environments, thus saving costs. These characteristics make JavaScript the most popular software development platform in the history of business computing. Moreover, JavaScript web applications have become the mainstay of line-of-business software, because the web technologies are ubiquitous, can readily be learned piecemeal, can be developed by disparate teams, and are easy to deploy on a large scale. These characteristics are absent in traditional software development platforms. Hence, there are tonnes of junior JavaScript web developer jobs available at present, and this trend will continue to grow, at least for the near future. While there are many CS graduates who work as developers, most coders in the IT industry do not possess a CS degree; they typically enter the industry either by self-study or via coding camps.
The less visible software work, though no less significant, occurs off the web: scientific and parallel computing; embedded systems; satellites and communications; radar and digital signal processing; military command-and-control; air traffic control; avionics, automotive, medical devices, consumer electronics, and appliance firmware; transportation logistics; real-time trading systems; artificial intelligence; autonomous vehicles; 3D games and simulators; compilers; the list is endless. This entire speciality is commonly referred to as system programming. Most programmers who work in these roles have STEM education, and many with either undergraduate or graduate degrees in CS. Unlike the line-of-business or social media software, even the entry-level jobs in these application areas require substantial theoretical knowledge. Perhaps this is why much fewer number of IT entrants take these roles. But these jobs pay much better by comparison, and they are also far more rewarding for the inquiring mind. And a lion’s share of technical innovations occur in these specialities. Given the nature and the importance of these specialities, much greater amounts of funds are invested therein, compared to enterprise software development. But of course the price of entry—that is, education, knowledge, skills—into these specialities is commensurately higher.
Every STEMer eventually rises to a technical leadership position. Engineers become engineering managers. Scientists become principal scientists. Mathematicians become department chairs. Programmers become lead technologists. Leadership positions in any STEM field require knowledge, experience, skills, creativity, vision, ethics, and business acumen.
A long, productive career endows a STEMer with knowledge, experience, and skills. Inner drive inspires creativity and vision in him. His innate nature informs his ethics. But to most STEMers, business acumen is not a natural talent. Some manage to pick up business skills along their career paths, but most do not get the opportunities. As such, many mid-career STEMers study part-time for the MBA, in preparation for a leadership role in the future. More education never hurts. But attending evening graduate school as a business unit manager, who also has family and social responsibilities, is not for the average person. More importantly, an MBA degree certificate, by itself, does not make one an effective leader of other technologists.
Years of experiences in academia and industry inform me that only hands-on STEMers with fresh skills can manage other STEMers effectively in-the-small and that only seasoned STEMers with business acumen can lead other STEMers effectively in-the-large. Do note that despite its apparent breadth, this carefully crafted statement is narrowly scoped.
Although it is not a common career choice, some high-achieving STEM graduates band together and form technology start-ups. Every successful Silicon Valley firm began life that way, and all large technology firms are founded upon the CS discipline. If your goal is to become the next tech billionaire before your 25th birthday, do keep these statistics in mind: every year, countless numbers of new corporations are formed in the US, but only about 5% of them survive past the fifth year; and, of those survivors, only about 5% retain their founders beyond the next five year. Even the big technology firms who retained their founders eventually replace the technology-minded founders with professional corporate managers in order to placate Wall Street. Although such actions raise the stock price in the short term, it stifles innovation in the long run.
There are obvious reasons why the top executives of massive corporations that design and build spacecrafts, airliners, ships, and cars climbed up the ranks of STEM. Corporate boards and Wall Street would do better to realise that large IT firms, too, prosper the most, when they are led by seasoned STEMers with vision of a better future, passion for the work at hand, and conviction for the greater good. That last bit—ethics—is ever more important for IT executives, for they alone hold the power to convert a thing of immense social value into an assault weapon against life and liberty, with just a flip of a boolean variable and without any accountability.
Today’s technology industry, and by extension world economy, is powered by the hard work of millions of developers. They have created, and are continuing to create, e-commerce sites, process automation systems, social media applications, and countless other useful technologies. Indeed, the software industry runs on the back of hard-working developers. And most developers in IT do not have a formal background in STEM.
A fair question therefore is whether a CS degree is useless in the IT industry. The answer is no. Many computer scientists work as developers in the industry. Although most of them would never have the opportunity to design a famous algorithm, they do create software that animate world economy. Their work is no less important to society than the computer scientists in academia. And the micro-decisions they make in their daily work is undoubtedly guided by the CS theory they absorbed in college. When they eventually rise to leadership positions, their macro-decisions will continue to be informed by their educational background.
But pursuing a CS degree is a significant investment in time, effort, and money. So, a youngster who is yearning to create the next great social media application or another addictive mobile game—but who has no burning curiosity for computability, computational complexity, formal languages, type theory, category theory, and other theoretical matters—is better off doing what he loves, without roaming about for several years in the forest of CS curriculum.
An interview is the opportunity for the potential employer and the candidate to evaluate each other whether they are a good match in terms of intellect, drive, culture, and personality. When a CS department at a university interviews a young candidate (freshly minted PhD) for a professorial post, the candidate is invited to give an in-depth lecture on his PhD dissertation, and all the professors and the graduate students are encouraged to attend. This lecture may last a few hours, and at the end of which there is a lengthy question-and-answer session in which everyone present is expected to participate. This is the chance for the hiring committee—the department chair and a few tenured professors—to evaluate the candidate’s teaching abilities. Then, the hiring committee meets with the candidate behind closed doors for several more hours, during which they evaluate the candidate’s personality, his research interests, and his potential for generating research and attracting funding.
In the not-so-distant past (until the onset of the dot-com era), many organisations in the industry follow a similarly lengthy interview process when hiring senior-level computer scientists. The few senior members of the hiring team would invite the candidate who passed the initial screening to a several-hours-long, face-to-face meeting during which the team treats the candidate as if he is a member of the team, presenting as interview questions genuine problems the team is facing, expecting the candidate to offer his own best-effort solutions thereto, based on his theoretical knowledge and practical experience. The correctness and the efficiency of the solution is often given lesser weight than the mental process the candidate employed in reaching his solution. The interview is generally comprehensive, collegial, and collaborative, like a typical design meeting amongst team members. Now, however, the interview process for developers has become mechanised and impersonal: a computer screens millions of resumes, and interviewers throw in the candidates’ faces a selection of questions taken from a popular interview questions book purchased from Amazon. If a candidate was lucky enough to have read the same book, he would give the exact answers enumerated in the book, and this would please the interviewers, immensely. Not only is this mechanised process childish, it is downright churlish to the candidate, and more significantly it is wholly irrelevant to the team’s actual work. Nevertheless, a young graduate hoping to land a junior developer job today has no other choice but to purchase a few interview questions books and memorise the answers, whether he understands them or not, and hope that those questions show up on the interview.
Like other industries, IT has every imaginable type of workplace politics. But one type of politics unique to IT is the awkwardness and the unpleasant that often arise when business-minded managers are placed in charge of technologists. Unlike law, medicine, and engineering where leaders are practitioners in the field, technologists in IT ordinarily work for non-technical managers whose priorities are schedule and budget. This management practice is endemic—nay, epidemic—in boutique-coding houses and IT-services firms; it is baked into the corporate culture of these companies. In IT, the majority of developer jobs are located in small companies started by businessmen. These companies usually develop business applications under commercial contracts or perform IT operations under government contracts. So, for many developers, business-focused contract work is their lot in life. Driving force in these firms is contract performance, not technical innovation. Given the nature of the work and the small, individual market share, profit margins of these firms are razor thin, so they survive contract to contract. The task of lining up the next contract, therefore, looms large on the management’s collective mind. These managers naturally admire, hire, trust, and promote only their own kind. As such, most developers do not get promoted beyond mid-level management positions in these types of organisations. Indeed, some of these companies have corrosive corporate cultures that view their technical staff as fungible resources. In such environments, developers do not last long enough to be promoted at all.
Academia, too, has its share of politics fuelled by rivalries. Intellectual rivalry and competitive spirit are good; they breed creativity and promote innovation. One of the best known academic rivalries was between two giants of modern physics, Feynman and Gell-Mann, at Caltech in mid 20th century. But academic politics is different in kind than the corporate variety. Corporate world is mobile; when politics grow particularly poisonous, a person can leave one company and join another in a sideway move. But a tenured professor is appointed for life, and if he flees to another institution, he loses his tenure and seniority. Squabbling professors are thus stuck with one another, essentially for life. As such, the potency of academic politics intensifies with time. But famous CS professors can escape, as many have done, to the corporate world, where a couple of additional zeros have been known to pop up on their pay cheques.
Many IT practitioners have backgrounds in non-IT STEM fields like science, engineering, and mathematics. Some even have backgrounds in liberal arts. So, a CS degree is not a prerequisite to enter this industry. But a CS education arms one with a broad, deep knowledge that may ease the path to a high-achieving IT career. The most important thing, however, is for the young person to pursue an education and a career that are aligned with his life passions.