[TOC]
This is a career guide for information technology (IT) practitioners: software developers, data scientists, system administrators, network engineers, technical team leaders, and junior technical executives. But it is also a checklist for college students contemplating whether to pursue a career in IT. Moreover, although the focus here is on the IT industry, much of the guidance offered applies to other STEM fields with equal vigour.
A career is a perennial pursuit, whereas a job is an ephemeral engagement. Clearly, this guide is not aimed at those who are merely seeking a job in IT. Fortunately for them, there are thousands of new publications that come out every month on Amazon targeting this large market of IT job seekers.
STEM is not a trade; it is a profession.
Both trades and professions require license to practise. Licensure guarantees that the individual possesses the requisite theoretical knowledge and practical skills, and prescribes the legal consequences of malpractice. The key difference between a trade and a profession is the level of formal, higher education required. A trade, like carpentry, pottery, or masonry, requires no higher education to enter as an apprentice, and it must be learned on the job under a master. A profession, like engineering or medicine, requires a substantial amount of higher education at an accredited university just to enter as an apprentice, then continue learning on the job under an experienced, licensed professional. STEM is a profession.
In the past, most IT jobs were occupied by highly educated mathematicians, computer scientists, physicists, and electrical engineers, who used mathematics and science to perform their duties. And they pursued that career life long, continuously progressing within their chosen professions. Today, though, the situation is quite the opposite; many in IT do not possess relevant educational background, and most people learn the skills they need on the job. And they hop from job to job, seeking higher pay to do the same work. In a way, IT has degenerated into a mere trade. Consider the process of building a computer. Conceiving, designing, and building the first electronic computer in the age of slide rule was hard; it required thousands of the most talented mathematicians, physicists, and engineers and the existential thread of a World War. But ordering Chinese-made components off eBay and snapping them together into a working computer is easy; precocious pre-teens have been doing that for years.
Fortunately, IT is not all board snapping, wire laying, button pushing, and copy pasting. There are IT specialities that require advanced degrees and, in some cases, even licensure: CPU design, medical device design, avionics design, air traffic control system design, etc.—the hard stuff that could kill, if you will. In terms of numbers, however, a majority of IT practitioners work as itinerant tradesmen, even if those positions have exalting titles, exorbitant pays, excellent benefits, and other accoutrement of a prestigious office.
If you want to have a career—not just a job—in IT, you should first acquire a bachelor’s degree in computer science, electrical engineering, physics, or mathematics, and preferably at least up to a master’s degree in a STEM field of your choosing. Once you have acquired the fundamental knowledge in mathematics and physics, as well as the advanced knowledge in your primary field, you can, if necessary, slip into any other STEM field to work, at least as an IT practitioner in that new field, and possibly even as a full-fledged practitioner therein. The reason is that much of STEM is intertwined at the highest levels, and the tie that binds is mathematics and physics.
Let us look at some examples. Robotics is a multi-disciplinary speciality that combines mechanical engineering, electrical engineering, computer science, material science, physics, biology, and mathematics. Artificial intelligence (AI) relies on psychology, neurology, electrical engineering, computer science, physics, and mathematics. CPU design requires knowledge in electrical engineering, mechanical engineering, physics, chemistry, material science, computer science, and mathematics. Quantum computing is an amalgam of physics, computer science, electrical engineering, and mathematics.
The inter-disciplinary connections are even stronger between mechanical engineering and electrical engineering. At the dawn of the Industrial Revolution, these two fields developed independently. But in the late 19th century, electrical engineering theories—circuit theory and filter theory in particular—matured quickly, propelled by the advances in physics. Today, the mathematical foundations of electrical engineering theories are taught in mechanical engineering, so as to analyse mechanical systems by analogy using electrical systems theories.
To sum up, if you have managed to acquire a solid foundation in physics and mathematics while studying as an undergraduate in any STEM field, you can progress to the graduate level in any other STEM field, with a modicum of extra effort to fill in the holes in your background. Conversely, to be able to reach the peak in a particular STEM field, you must have at least a fundamental knowledge of other, closely related STEM fields. What this means is that one-trick-ponies do not reach the pinnacle of their careers in STEM.
No car company has ever hired a restaurant manager to lead a team of mechanical engineers designing the next generation engine. In STEM, only experienced practitioners are qualified lead other practitioners. A leader in a STEM field must possess both theoretical knowledge and practical experience.
This, however, is not the case in IT consulting, especially in the specialities of service delivery and web development. In a small firm, consulting projects are proposed, won, and managed by the founder and his close associates—usually his golfing buddies—who are all entrepreneurs. In a large consultancy, projects are run by managers who completely lack technical knowledge and skills, and sometimes even management experience. But this lot has amassed reams of management training certificates and tonnes of experience pricing projects and cutting costs. There are many reasons why this strange situation exists in the IT industry, but it is beyond the scope of this article.
The fact remains though that only an experienced technologist can be effective in leading and managing other technologists. Innovative companies recognise this, and they employ experienced technologists as technical managers. So, as a STEMer, if your aim is to move up the management ladder, you should join companies that do scientific research or technological product development, instead of those that perform service consulting or web development. But you must first accumulate theoretical knowledge, technical skills, and hand-on experience. Also know that large projects are fundamentally different from small projects, even if the technologies involves are similar. It can be quite overwhelming, perhaps even counterproductive, for a fresh college graduate to dive straight into a massive, multi-team project as his first engagement. So, when you start out in the industry, try to join a small team working on a small project. Then, in your mid career, seek out opportunities to work on larger, well-managed projects. By then, your experience will guide you in choosing the right environment that fits you the best.
The lack of a graduate degree can sometimes become a hurdle on a management career path. Most people with management mindset eventually get an MBA. But to me, a law degree and a law license are much more useful, even if this path is much more difficult and resource intensive. I am, of course, biased on this count.
Humility is a necessity.
It is the corporate norm, these days, to 12 hours or more a day in the company of our colleagues. If we discount for the sleeping hours, we spend more time with our coworkers than with our families. So, collegiality and camaraderie are essential not only for workplace productivity but also for psychological welfare. We humans have an innate need for acceptance and respect. Respecting others is the surest way to earn the respect of others. And without humility, one is incapable of respecting others.
Being humble is not the same as being feeble. You are hired for your expertise, so you owe to your employer and colleagues your best judgement, offered in forceful and forthright manner. But collegiality does demand that you proffer reasoned arguments and that you follow proper etiquettes, when communicating your opinions and recommendations.
On the other hand, do not be imperious. People do not tolerate one who craves constant attention, who always dominates discussions, or who always insists that everyone adopts his view. Let reason, process, and etiquette guide you.
Another personality trait that grates people is prickliness—thin-skin. We all make mistakes from time to time. When our teammates offer constructive critiques, we should accept them as such instead of being defensive. And more importantly, we should resolve to never repeat the same type of error.
And to be a good colleague, you must not procrastinate. Procrastination is a common personality quirk. Although it may resemble laziness, hardworking people, too, procrastinate. There are a number of procedural techniques that can act as a prophylactic against procrastination. I discuss these techniques, later.
An oft-overlooked personality trait in a high-pressure workplace is generosity. Generosity is a measure of seniority. If you have the capability and capacity, you should offer your support to a struggling, junior teammate. And you should publicly recognise a teammate’s praise-worthy action in a genuine, non-condescending manner.
In due course, every technologist accumulates enough experience to ascend to a leadership role, first leading a team, then a division and, eventually, the whole company. This is the management career path in every science and engineering outfit, and a few technology firms. If this is your planned career trajectory, you will gradually have to transition from managing technology to managing people, then to managing large groups of people. So, it behooves you not to be not a jerk, from the very start.
causes—Politics, a bad thing, naturally emerges from collegiality, a good thing. This sounds paradoxical, but it is the nature of human beings.
Employers see the intangible benefits of camaraderie. As such, companies regularly hold picnics and other events for the employees. In a small company where everyone works under close confines of one roof, such social events are particularly effective. But in a large organisation, company-wide social events are impracticable. So, large companies necessarily have to organise the employees into divisions, branches, units, and teams. To be effective, a team must be cohesive. But there are potential downsides to team cohesion.
Cohesive teams tend to suffer from isolation. While cohesion is a good thing, so is diversity—especially the diversity of views. But tight teams naturally admit only those who think, speak, and act like themselves. This nature robs the teams of the opportunity to diversify and, consequently, they self-select their way to become insular.
Small, insular teams invariably exhibit the herd mentality, where team members refrain from doing and saying what is necessary, in fear of offending colleagues and damaging team cohesion. Many a bad decision has been made by the herd.
A side effect of herd mentality is peer pressure. It is exceedingly hard for a team member to leave the office at the close of business—say, to attend to a family matter—when all his colleagues are still hunched over, pecking away at the keyboard. This invariably brings forth personal complications and psychological complexes.
Perhaps the most pernicious effect of team cohesion is phalanx forming: us-against-them, hands-off-ours, and other defensive tactics. Such tactics are suited to combat, not collaboration. In the most benign case, porcupine teams diminish productivity of the organisation. In the worst case, they deplete morale throughout the organisation.
It is the responsibility of all employees, especially the leadership set, to spot and stop the bad consequences of a good thing—this team spirit. There are no black-letter rules that organisations can follow. Each instance must be dealt with, case by case. The best prevention is for the leaders to monitor continuously and to intervene immediately. And this collaborative mentality must be woven into the corporate fabric.
The opposite of cohesion is discord—strife amongst the teammates brought forth by bravado, competition, resentment, envy, and other human frailties. Perhaps the commonest frailty that excites discord is laziness. One lazy team member, if left unchecked, will breed resentment. In fact, laziness is contagious, because if one team member can get away with it, others will copy his behaviour. A lazy team always falls back on blaming as the primary strategy. So, both cohesion and discord give rise to politics.
cures—Politics is a constant presence; only the intensity varies. The leadership—from a team leader up to the chief executive—can be both the exciter of politics and the suppressor thereof. I address the responsibilities of the leader later. Here, I describe what a team member can do to mitigate workplace politics.
By its very nature, corporate politics is something you discover only after you have joined the organisation. So, as a new member, you should, at your earliest opportunity, observe the dynamics within the team as well as the team’s interactions with external units. That should not be taxing, for opportunities to observe politics are abundant and ever present. The “earliest opportunity” may even be during your interview. Once you have a general sense of the factions, you can either join one or avoid them all. If you are a genial and collaborative, I strenuously recommend the latter strategy. Nothing good has ever emerged from corporate infighting.
One good shield against political entanglements is diligence. If you do well what you were hired to do, you are less likely to be ensnared into the swirls of politics. Surprisingly, your diligence can also paint a target upon your back. Once the lazy lot realises that you are making them look bad, you have just entered into an untenable $1\ v. N$ combat.
To avoid such unpleasantness, the only realistic thing you can do is to meet as many team members as practicable during your interview and try to assess their personalities. This, however, is no guarantee, since it is virtually impossible for you to grasp the true nature of people in just a few minutes of meeting.
Although it is rare, there are some extreme cases where a team dynamic resembles that of a prison atmosphere: join a gang or perish. In that case, you might consider resigning. The law prohibits employers from impeding employees’ mobility.
A STEM workplace requires one to possess many seemingly contradictory personality traits: congeniality, resilience, adaptability, tenacity, restraint, dexterity, assiduity, etc. When personalities do not gel, the team environment degenerates into a harsh social experiment. But when the essentials are balanced, the workplace can be not only professionally rewarding but also personally fulfilling.
Communication is duplex.
Those who were raised on mobile devices and social media tend to have a much shorter attention span than that of the prior generations. The young, tech-savvy IT crowd seems to suffer from this ailment the most acutely: many cannot concentrate on one topic beyond a few minutes. Indeed, most people in IT today do not read, neither literary works nor technical publications. Instead they acquire “knowledge” by watching 10-minute YouTube instructional videos (each with a VPN commercial interlude that takes up several minutes).
Look at it this way: every time you watch an influencer’s video, you lose several minutes of your life rediscovering facts that you already know about a product you already decided you would rather not buy. You should, instead, have spent this time learning, exercising, sleeping, or relaxing with your family.
A fast-context-switching mind is well tuned for consuming social media content, but it is ill matched for absorbing structured business information: corporate policies, regulatory compliance guidelines, work instructions, etc. In recognition of this modern phenomenon of short attention spans, many corporations have supplemented their lengthy documents with short videos. But this move only worsens a troubling trend: if a writing is longer than a tweet, few people today bother to read it—including a guide like this one.
But to be an effective communicator in the workplace, you must first be able to absorb information, tonnes of it, in very little time. That means you must learn to read fast and, more importantly, critically. I provide several tips in my article on reading and writing techniques for STEMers.
Do note that much of business communication is in spoken form, usually over the phone. So, you must also learn to listen attentively. And when you cannot see the speaker’s face, you must work harder to sense his sentiment by his tone and diction.
The best advice I could offer is to absorb more, exude less.
First of all, there can be no output without some input. So, listen, read, and think, before you speak or write. Always consider in advance the possible effects of your speech, whether spoken or written. In the heat of the moment, a speaker may fail to distinguish between refutation and denigration. But to the listeners, the difference in tenor is a world apart. So, think before you speak, and edit before you send. Shunning brashness is generally a good strategy in business communications.
Volubility should also be avoided, especially when communicating with executives. Senior executives are busy. As a matter of course, you should provide them only the information that they need to make big decisions.
Equivocality is similarly inappropriate, because an ambiguous statement is neither comprehensible nor actionable by the recipient. Definitely avoid making vague statements in official communiqué transmitted to the customers or the media, for there may well be legal consequences.
Likewise, irrelevance is equally bad. If your superior asks an urgent question on a specific matter under your control, your answer must be succinct and it must be responsive to the query. And you must suppress the temptation to tack on a report about a different matter. Leave such irrelevant information for the omnibus status reports.
Omission of a critical detail from status reports can be very harmful, both to your employer and to your customer. A critical detail is a piece information necessary to make decisions or that which can negatively impact the project, the employer, the customer, or the public.
What is even worse than omission is obfuscation—sugar-coating a bad news or hiding it under a mound of irrelevant, but feel-good, information. It is a common knowledge that obfuscation has dire consequences. Yet, many people, especially those in leadership positions, are frequently seduced by it. Know that a clever turn of phrase cannot diminish the impact of a negative outcome. Identifying the source early, seeking support when necessary, and solving the problem collaboratively is the only reasonable strategy.
Today, organisations rely on text message, email, blog, and many other forms of written communication. The information exchanged in the context of your employment—even those messages that originated on your personal device—belong to your employer. If a legal dispute erupts, say between your employer and a competitor, that company may well demand that your employer produce all potentially relevant messages, including those on your personal device.
Not long ago, employers issue corporate communication devices—two-way radios, two-way pagers, etc.—to employees, because such real-time communication services were rather expensive. But today, mobile phones can do all that and more, and a typical phone plan includes a generous data service. So, employees now use personal devices to conduct the employer’s business. It is quite likely, therefore, that your private messages, which are beyond the context of your employment, may unintentionally get caught up in a dragnet discovery, pursuant to a law suit in which your employer is involved. So, understand the boundaries of your privacy, when you use your personal device on the job.
Programmers must recognise that their code is a form of communication. In the distant past, the code was the means by which programmers communication with the hardware. Today, however, the code is the primary means by which programmers communicate with their colleagues. It is every programmer’s responsibility to write both code and the accompanying comment, cleanly, clearly, cogently.
Here is a little warning: the more you say, the higher your error rate.
Wander the wonder of the unknown.
A competent STEM practitioner must be proficient in both theory and practice. Field-specific theory and mathematics (mostly analysis) must be learned at the undergraduate level. Those who progress to the graduate level will need to take more advanced mathematics (including abstract algebra) and more specialised theory.
In computer science, for example, undergraduates study calculus (ODE, PDE, and the like), basic physics, basic chemistry, and introductory programming in the first year, then move up to the core courses, such as discrete mathematics, logic, algorithms analysis and design, relational theory, communication networks, computer architecture, operating systems design, programming languages, system analysis and design, and basic complexity theory. A person with a bachelor’s degree in computer science from an accredited university may not have designed and implemented a massive, cloud-based enterprise system, but he is deemed to possess the theoretical knowledge and the practical skills to be immediately productive as a junior team member—at least in theory, that is.
Graduate computer science studies include specialised practice-oriented courses, such as CPU design, computer graphics, artificial intelligence, natural language processing, digital signal processing, and embedded systems design, as well as theory-oriented courses, such as category theory, advanced complexity theory, type theory, and programming language design. Some people with a PhD become an associate professor at an institution different from their alma mater; many enter the IT industry as senior scientists; and others continue on to post-doctorate studies at a research institution.
If your goal is to have a career in IT, you need a bachelor’s degree in study computer science, electrical engineering, physics, or mathematics. It is advisable that you either go straight to graduate school in a STEM field of your interest, or attend graduate school part-time, while working as a junior technologist. Given the pace of development in STEM over the past few decades, STEM practitioners should have at least a master’s degree. This is why I recommend completing your graduate studies, well before professional, social, and familial responsibilities pile up, later in life.
Do note, though, that a great many IT jobs today are not careers but mere trades: project management, web development, data processing, network wiring, data centre monitoring, help desk, etc., where no deep theories are needed to do the job and much of the skills needed can be learned on the job. Indeed, many people who fill these jobs do not have higher education—they do not need it. IT works that do rely on theory are CPU design, digital signal processing, digital image processing, programming language design, algorithm design, operating system design, simulation, control systems, artificial intelligence algorithm design, and so on.
I would like to bring your attention to this very important point: as a STEMer, you live and work in the border between the known and the unknown, so learn to be comfortable with the unknown—specifically, converting the unknown into a known—and be a fearless explorer.
STEM work in an industrial setting requires both theory and practice. In the IT industry, practical work always involves programming, to varying degrees. Popularity of programming languages wax and wane with time. Unlike in the past, one language no longer remain dominant over a person’s career. And each application domain has at least a handful of commonly used languages. It is impossible for one person chase those thousands of languages. But there is a simple strategy to master programming languages: master C, Smalltalk, Standard ML, and SQL.
C is the ultimate procedural language, and the favourite of those who work with operating systems and embedded systems. A good C programmer can work on any project that require direct manipulation of the underlying hardware. And most modern languages use its block-structured syntax.
Smalltalk is designed for non-technical users, so it is easy to learn and natural to use. It is the mother of popular, modern objective languages, including Objective-C, Java, C#, Python, JavaScript, and Swift. If you know C and Smalltalk, you already know Objective-C, and you can pick up C++ in no time. And despite being much larger and more complicated languages, Python and JavaScript are quite close in nature to Smalltalk. So, Smalltalk knowledge will make learning these newer languages relatively easy.
Standard ML is the progenitor of all modern functional languages, including Haskell, Agda, Coq, Lean, and others. New objective-functional hybrids—OCaml, F#, ReScript, TypeScript, Rust, Swift, Kotlin, Scala, etc.—all share the ML DNA, so knowing ML will make it easier for you to learn those newer languages. But because ML is the oldest of them all, it is the simplest of this lot. Moreover, ML has a long history as the academic language taught in computer science curricula round the world, so there are many well written, theoretically sound instructional books on ML. Hence, ML is the best starter language to learn functional programming, properly.
SQL is the only relational language that is extensively used in the industry. Every database platform supports it. Since all enterprise-scale software systems use a database as the persistence medium, SQL is a must-know language.
I would add Scheme as a good-to-know language. Scheme derives from Lisp, the world’s second oldest high-level language and the first functional language. Although the majority of IT practitioners today do not know Lisp, XML is very much like Lisp, only with uglier syntax and without homoiconicity. Every XML configuration file can be replaced by a Scheme programme. In that sense, a Scheme programme is a configuration file that can run itself.
In addition to languages, you must know software design principles: procedural, objective, functional, and modular.
Procedural design employs procedures, a collection of statements, as the primary abstraction. A programme invokes many procedures to manipulate the global data structure.
Objective design uses as its main abstraction the concept of interacting objects. Objects hold their own internal states, and these states evolve as objects interact with one another by sending messages to and fro.
Functional design employs functions, expressions with inputs and outputs but no internal states, as the primary abstraction. A functional programme is a composition of many small functions. There is no global data structure in a functional programme. Function that make up the programme transform the input data, step by step, until the desired output is achieved. Functional paradigm is the one that is closest to mathematics. Hence, IT practitioners with mathematical training prefer functional languages.
Modular design can be used with procedural, objective, and functional paradigms. A module is a collection of related data and code. A typical module maintains an internal data structure which can be manipulated via the procedures or functions exposed by that module. Hence, a module is an abstraction mechanism that hides the representation details irrelevant to its published functionality. Modules in objective languages, however, are somewhat different in character. Objects provide an even finer-grained representation-hiding mechanism than modules of procedural and functional languages. So, in objective languages, a module is used to house closely related objects, whose interactions are deemed low-level details irrelevant to the published functionality of the module.
It is not an exaggeration to say that if you have mastered, I mean truly mastered, the concepts mentioned above, you will be an effective programmer in any language—that is, until quantum computing becomes commonplace and you must then transition to a completely different way of programming.
Collect, analyse, then synthesise.
There is a software development philosophy called extreme programming (XP). XP emerged in early 2000s, as a reaction to the rigid waterfall model of the 1970s. Indeed, XP is but one among many similar reactions, all based on the 1980s concept called spiral model.
The waterfall model has been practised in engineering for millennia, probably dating back to the time when the first city states sprang up in Sumer. This approach requires every stage of the project be fully completed, before progressing to the next stage: analysis, design, implementation, testing, operation—in that order, in lockstep. So, civil engineers take pains to avoid as much mistake as humanly possible in their analysis and design, before they lay the first brick or span the first beam. They are able to work in such a disciplined way, because their field has had six thousand years to learn from some truly monumental mistakes.
Modern software development, as we know it, came about only in the 1960s, so it is relatively immature. Hence, software developers’ methods, too, are relatively immature. At the present maturity level of the field, it is impossible for a programmer to implement, on the first try, an error-free programme longer than a few tens of lines, and it is impossible for a published programme longer than a few tens of thousands of lines to be free of dormant errors.
The way we compose novels and programmes is very different from the way we build bridges and computers. So, by the 1980s, the IT industry recognised that the traditional waterfall model was inappropriate for software development. As a result, the spiral model was conceived. This approach promotes building software in increments, each increment, called the iteration, lasting just a few weeks. An iteration is a complete, but small, waterfall process. That means every short iteration yields a working programme, albeit with only a small set of usable features. This iterative delivery allows the users to be involved in the development process and gives them the ability to correct the course of development before the project goes awry.
This iterative development concept has proven so successful that it spawned numerous offsprings. Right or wrong, these spiral, iterative processes have been lumped under the label “Agile”, although the Agile Method, like Extreme Programming, is a separate method. The key innovation of the iterative Agile process is to recognise and accept the fact that in the early phase of the project, the software development team does not, and indeed cannot, fully grasp the problem they were tasked to solve, and that they must probe deeper into the problem, iteratively. Agile frees the programmers of the guilt of making small mistakes early on, provided these small mistakes are immediately remedied in the subsequent iterations. But Agile does not release the programmer of his professional responsibility to act with diligence and conscience.
It is generally known that hardware mistakes are costly to fix, and it is generally believed that software mistakes are cheap to fix. This was true, when programmes were only a few hundred lines long and each written by one programmer. This is most assuredly not the situation today, when a typical software product development process is comparable in complexity to a typical hardware product development. When the spiral model came about, programmes were smaller than modern programmes, by orders of magnitude. On that scale, a couple of team members pair-programming under the Agile process was the recipe for success. But those happy days are gone forever. So, before you reflexively apply this “highly successful” Agile process to your modern projects, you must closely re-examine the assumptions and the conventions from whence Agile sprang. I would venture to say that your large, multi-national, modern software project looks nothing like a typical spiral project of the 1980s or a typical Agile project of the 2000s; in fact, the complexity of your project is likely closer in character—technical, managerial, financial, political, scope, and ambition—to the renowned Anglo-French project that created the Concorde.
Today’s software projects are fundamentally different from Agile projects of two decades ago. Scope is now more expansive, technologies involved more complex, and cycle times more contracted. The “ship it” mentality, brought on by the intense competition, has robbed the teams of the time necessary to repair and refactor its earlier errors in assumptions and implementations. So, blithely pumping data from API to API without understanding the problem, hoping to fix the mistakes later, is no longer the right approach. You now need both the discipline of the waterfall model to cope with the immense complexities, and the ductility of the Agile process to deal with the fast-evolving requirements. Analyse before synthesising, and balance control against agility—look before stepping. Recognise that applying Agile indiscriminately will only make your process fragile.
A very dangerous misconception in modern software development is the overly optimistic view of, and false faith in, software components. Programmers believe that well designed components are fully and automatically reusable and that no one needs to understand how such components work. This is evidenced by the general lack of understanding amongst software developers of the lower level concepts: how the CPU works, how the operating systems work, how the networking protocols work, etc. In truth, even the best designed software component is specialised to a particular application domain, and using it effectively requires the programmer to understand not only its external structure but also its internal behaviour, at least to some extent.
A related misconception is that understanding of the behaviour of a large, complex system emerges automatically from the understanding of its constituent components. Gestalt psychology has long shown that the opposite is true and, hence, the oft-repeated aphorism, “The whole is greater than the sum of the parts”. The success of component-based architectures on the small scale in the 1990s had led astray modern software architects: they now assume that so long as component interfaces are well defined, disparate teams can work independently on separate components and assemble a large system therefrom, without the need for a central coordinating authority. This is nonsense. Regardless of how cleanly separated the components are, it is still necessary for a few people to understand the behaviour of the entire system, and in this group must lie the authority and the accountability of the whole. Cloud platforms and micro-service architectures are just tools; conscientious workers are the real solution.
There are two diametrically opposed problems related to analysis: paralysis by analysis and fear of the unknown. Paralysis by analysis occurs when the team cannot resist to urge to scurry down the tunnel of every possible scenario, thereby stalling progress altogether. Fear of the unknown, which is a natural human emotion, often compel programmers to avoid learning the essentials about the problem, such as the nature of the business domain, the aspirations of the client corporation, the needs of the end users, and so on. In an acute case of the fear, a programmer may, against all reason, refuse to learn anything beyond what he needs to do his job. Both the paralysis and the fear diminish productivity.
The team manager can effectively prevent analysis paralysis through weekly team meetings and individual coaching. But you, the individual programmer, must cope with your fear of the unknown. The only way to eliminate your fear of the unknown is to convert that unknown into a known—read, listen, ask, study, and document what you learned.
Analysis without observation is speculation, and synthesis without analysis is chaos. Observe, analyse, then synthesise. This is the order as natural as the generational order: grandparents, parents, then children. And just as the lives of the generations overlap, so too must observation, analysis, and synthesis.
At the inception of a project, a typical web development team charges boldly up the implementation hill patriotically waving their XP flags, and without their flanks and the rear guarded by requirements analysis and user interviews. Up that hill awaits defeat, well concealed. And as foolhardy charge on the battlefield kills, many a project has died, boldly. Yet, web developers persist in their boldness—read foolishness.
There are different kinds of failures in IT: project folded without delivery (not working); project delivered a barely functioning system (partially working); project delivered a system that meets the current requirements, but is not extendible due to poor design and implementation (working but not extendable). In the distant past, non-delivery was quite common. Today, though, software development methods and component technologies have become advanced enough that no one starts from scratch, so everyone is able to deliver at least a partially working system. A not-so-uncommon scenario is that a large system is so full of bugs at launch that it is almost unusable—the Obamacare site, for example. And most large systems that have been in operation for longer than a decade has been so poorly designed initially and so haphazardly modified subsequently that the system, as a whole, is no longer comprehensible even to the experienced programmers on the project. A system that barely works is as useless as that which does not work. And a system that cannot be understood by its programmers is bound to rot away and become useless, sooner or later.
To avoid these failures, every generation of technologists who work on a large, long-running IT system must improve that system and to record the pertinent institutional knowledge for posterity. Understand this: code is not comment, and comment is not documentation. A documentation is a roadmap of the whole work. A comment is an explanation of some important detail for future programmers who must read this section of the code. And code is the investment the programmer leaves behind for his successors.
A day-shift doctor treating a critically ill patient must record in detail his treatments and the patient’s responses, so that the night-shift doctor can fully assess the situation and provide continued care. But a plumber need not leave cute little notes hung round the pipes for his successor, because the next plumber can assess what he needs to do just by examining the state of the pipework. Programmers today behave very much like plumbers. But what programmers should really do is to put in even greater effort than that of the doctors in recording the work. A doctor’s interactions with a patient affects that one patient; a programmer’s interactions with a system affects life, liberty, and property of many users. One who considers his work a profession ought to care about the welfare of those who are connected with, and those who are affected by, his work, as well as the legacy of his work.
A programmer will face many uncertainties through his career—unexpected shifts in priorities, upheavals in market conditions, sudden obsolescence of established technologies due to the emergence of a new one, etc. But the following are the few certainties: he must read someone else’s code; he must write his own code; and he must debug code, both his own and that of others.
Programmers should not be forced to read code to understand what the system is supposed to do—they should be given up-to-date documentation for that purpose. Sadly, the culture of documenting one’s work has all but evaporated in the IT industry. And what little documentation exists is outdated, since no one bothers updating the document, when the system is modified. So, every programmer is obliged to read the code to get a sense of what he needs to do.
Today, many teams do not even write test cases and documents anymore. The use of cloud and other distributed technologies prohibit, or at least make it prohibitively expensive, to write test cases that can expose all potential problems in timing and synchronisation among distributed components. Typically, a team’s development environment is nothing like the production environment, and it is impracticable to establish a test environment that meaningfully emulate the dynamic nature of the production environment. The consequence is that most programmers debugging existing code do not understand the system and its environment, adequately. Politics makes things much worse: some of the bugs are in the components owned by other teams, and those teams jealously guard their components. So, the debugging programmers are forced to implement workarounds, instead of collaborating with other teams to eliminate the source of the bug.
In general, junior and mid-level programmers do not create new code; they are pushed into debugging existing code—the task senior programmers detest. In the not-too-distant past, many programmers subscribe to the principle that one should not commit to the repository a code that does not pass all the tests. Now, though, the situation has devolved into this sad state of affairs: senior programmes generate bugs because they do not fully grasp the complexities that inhere the system and its environment; junior programmers spend their days debugging, also without adequate understanding; and because mid-level programmers, too, lack the requisite understanding, they are unable to offer guidance and assistance to junior programmers.
The many causes of this disease are well understood, but no one is working on the cure, because there are no profits in such a cure. The modern software development culture is to add fancy features, non-stop. This is caused by rival companies competing for bells and whistles, instead of for stability and usability. The market pressure constantly to release new features has robbed the teams of the time to analyse before synthesising. Standing on the principles and steadfastly refusing to rush the job may lead to the loss of eyeballs, especially in the social media market. This scenario is unsustainable.
And modern technology users have come to expect applications, even business systems, to be visually attractive and procedurally entertaining. The traditional rule of thumb in software development is that 70% of the code is devoted to the user interface (UI); 20% to the infrastructure such as security, privacy, storage, deployment, and so on; and only 10% to the application logic. But this irrational emphasis on fancy features means that the complexity of the systems grow exponentially and abruptly, without adding features that truly benefit the operation of the business. Business executives, technical managers, and technologists working on enterprise applications should understand that such applications are fundamentally different from social media applications. Enterprise applications are now developed using the open-source technologies published by social media giants, and are developed and deployed identically to social media applications. Programmers want to burnish their resumes. Executives want to show off fancy features to the public. And manager, lacking background in technology, are unable to rein in the bullish programmers who always want to play with new technologies. This is a fatal spiral down into the depths of complexity.
Another problem creator is the industry’s propensity to take delight in constantly creating innovations that are no more innovative than the ones they aim to replace. Just look at the partial list of the existing web frameworks, client side and server side. There is no technical guidance for teams to choose a framework that suit their needs, because all these frameworks are essentially identical. So, teams choose the most popular framework, regardless of its suitability to the task. And a few weeks later, another framework ascends to the top spot, and everyone scrambles to retrofit the new framework. This is unproductive, wasteful, and foolish.
Also, the art of troubleshooting is now dead, d-e-d dead, in software development. The traditional way to troubleshoot a failing system is to place the system under test, analyse the failure condition, isolate the subassembly where the failure manifests itself, identify the possible causes for the failure, eliminate the non-causes one by one, find the failed component, replace it, and retest the whole system for a final check. This was how radios, televisions, refrigerators, cars—all things—were repaired and reused. In modern practice, however, a car mechanic will replace the whole large-screen display, even if only the power switch is the only malfunctioning component. A hardware system can be repaired by replacing the failed component with an identical, working component. This is so, because hardware components have predetermined operating lives, and they are fungible. But this is certainly not the case with software. A failed software component cannot be replaced, but must be fixed, in situ.
Moreover, in a hardware system, like a car, it is painfully obvious when there are nonsensical global interactions. No automotive engineer could commit a design mistake—like a fuel gauge that interferes with the operation of the fuel injection unit—and expect to keep his practice license. When lives and limbs are involved, the consequences of a malpractice can be harsh. But in the unregulated world of enterprise software development—billing systems, employee benefits systems, inventory systems, shipping systems, etc.—it is extremely easy to commit a foolish design error, like accessing a piece of data in the billing system directly from the shipping system. When there are no hard protocols and no severe consequences for breaching them, people tend to take shortcuts, especially when they must work under time pressure and they do not understand what they are doing. And given the nature of code, such lazy shortcuts and design flaws remain well concealed.
It is time for the industry to recognise that the cost of software mistakes are now just as dear as those of hardware mistakes. Hence, software practitioners must adopt the practices of hardware engineers: thorough analysis, careful synthesis, proper documentation, redundant components, multiple failure checks, etc. A programmer should handle his code and his development tools like an electronic engineer handles his product and his expensive test equipment—with care and due respect.
On a different note, I have noticed, through the decades, that young programmers are particularly susceptible to procrastination. Causes vary: a few senior programmers are capable and confident enough to complete the work seconds before the deadline; some junior programmers suffer from the foreboding fear of the unknown, and no one offers any guidance; many mid-level programmers suffer from analysis paralysis. In whatever form, procrastination is unprofessional.
The most effective prophylactic against procrastination is planning and following through. Procrastination is an entirely normal behaviour. If you see a pile of dirty dishes in the sink, stench and all, you feel disheartened and are not motivated to clean up. But if you make it a habit always to keep the sink clean, you are motivated to clean up every time a dirty dish appears in the sink. Discipline goes the distance.
There are necessities, then there are niceties. When planning your work, prioritise necessities over niceties. A plan of attack is needed, for any task that requires more than a few minutes to complete. That means you should start your day be creating a rough outline for the whole day and a slightly more detailed plan for each major task to be performed during the day. A short-term plan like this needs neither be elaborate nor ceremonial—it could be done in your head in just a few seconds, and be held there for the duration of its execution. Sometimes, you will have to alter your plan, because of urgent meetings or unexpected new priorities; do not fret. The important thing is to think about the overall work and to have a mental roadmap of it, before you begin working.
Too many XP aficionados view disciplined analysis as a waste of time. They confuse coding with progress. In reality, aimless coding is waste disguised as productivity. If you find yourself unconsciously engaging such behaviour, recognise it as a frailty and remedy it. Get a pen and a pad. Always start by writing or sketching your ideas on paper, away from the keyboard. If you cannot explain in plain English your solution to another person, you are not ready to code. Start typing, only after you feel like you have grasped the immediate problem. Even then, you will find yourself confused about edge cases. Whenever you feel a wave of confusion rush over you, have the humility to tear yourself away from the keyboard and to seek clarity on paper with pen in hand.
Perhaps the most insidious, and the least obvious, consequence of under-understanding of the problem is the creation of an over-complex solution. A tradesman can satisfy himself with a job barely done, but a professional must always rise to an elegant solution. “It works, doesn’t it?” does not work for a professional. A clever, complex solution shines a glaring light upon the inadequacies of the approach and the incompetence of the problem solver.
Unethical conduct has a distinctive stench.
What constitutes unethical conduct depends on the profession. Medicine and engineering are regulated STEM professions and, consequently, have the highest ethical standards. Each jurisdiction oversees the professional activities of doctors and engineers, and metes out punishments commensurate with misconduct and malpractice, ranging from reprimand and fines to loss of license and imprisonment.
Although other STEM fields are not professions subject to government regulation, each has a code of conduct either steeped in tradition or sprang from consensus. For instance, it is unacceptable for a scientist to plagiarise another’s work or to forge experimental results. A scientist who commits such a misconduct instantly becomes unemployable.
Traditionally, there was no need to regulate the professional conduct of mathematicians, since it would be impossible for anyone to convince the community to accept an erroneous proof. But in recent times, proofs have become intractably large. Mathematics is accretional: each new proof relies on many existing proofs in various sub-specialities. And mathematics has become very modular: each sub-speciality understands only the statements of the theorems produced by other sub-specialities, but not the proofs. This opens up opportunities for some to produce massive, convoluted proofs that others could neither verify nor disprove. What is STEM if we cannot rely on the mathematicians? To counter this dangerous trend, some mathematicians are looking to the automated theorem provers—like Agda, Coq, and Lean, which are well and truly in the domain of computer science—to formalise (mechanise) the foundations of mathematics so as to verify the proofs in higher mathematics. Theoretical computer science is a branch of mathematics, so it is governed in the same manner as mathematicians.
But what of the technologists in IT: programmers, system administrators, network administrators, database administrators, and so on? Well, there are no regulations that govern the professional conduct of this lot. So, if you are pursuing a career in IT, you must tread lightly.
In any profession, there is a dividing line—thin as it maybe—between morals and ethics. Some ethical conduct of a professional may well be deemed immoral by a moralist. For example, a defence lawyer representing a murder suspect may have a sense that his client is guilty, but under the code of conduct of the legal profession, he must ardently and diligently represent his client, and exploit every legal, tactical, and procedural error committed by the prosecutor. Likewise, a corporate executive must, under corporate law, maximise the corporation’s profits, even if the corporation’s actions (that is, his decisions) harm the environment or degrade his customers’ privacy, so long as such actions are permitted by local laws. This can be a difficult concept to countenance, especially for a morally upright person. But it is vital that you, as a professional, possess the fortitude and do not become disillusioned or distracted. Above all, do not lose your moral footing and set yourself adrift in the morass of corporate ethics and profits. A STEM professional is still responsible to voice his concerns to his superiors, whenever a lawful corporate action could lead to the denigration of a long-held cultural or moral standard.
In every large IT organisation, these days, there are strategic turf wars and tactical political sniping, within a team, between teams in one organisation, and between teams from partner organisations. Indeed, it is now impossible to build an IT system without investing inordinate amounts of time and effort in courting political alliances. While playing politics is not unlawful, it is downright unethical. It harms the organisation, its employees, and its customers. But such wars rage on. As an IT professional, you would do well to steer clear of politics. But that is easier said than done.
The worst kind of turf war is between teammates. Many programmers, through the ages, have shown the propensity to write uncommented, convoluted code, sometimes to put up airs of superiority, but mostly to protect their job security. As a programmer, your employer pays for your time and, hence, your employer owns your work product. Giving your employer incomprehensible—pronounced, “useless”—pieces of code is highly unethical, both as an employee and as a teammate.
In IT today, many practitioners are preoccupied with resume burnishing, and not with performing the work. Programmers, for example, live in constant fear of being left behind by the so-called “advances” in technology, namely the perpetual reinvention of the wheel. This lot is prone to chase the latest shiny framework, necessary or not, beneficial or not. Such conduct wastes corporate resources, and most importantly, investors’ funds.
A similar ill is the curiosity constantly to play with new toys—the dilettantism. Staying on the bleeding edge has serious negative consequences. The bleeding edge technology that is the darling of the industry today can become passé tomorrow. Conversely, stodgily holding on to the old technology can rot a system. Judgement is an area where a professional earns his keep. So, make your decisions based on evidence and need, not on whim and fashion.
Perhaps the most detestable, not to mention unprofessional and unethical, conduct in IT is the way government contract programmes are proposed. A large government IT consultancy has dedicated proposal division. From a resource reuse perspective, it makes sense for the corporation to have a team of proposal specialists writing, offering, and negotiating government contracts. But a typical proposal team is made up of business-minded writers with no concept or experience in IT products and services. To make matters worse, the overarching regulation, the Federal Acquisition Regulation (FAR), which was enacted in the 1980s at the birth of IT consulting, irrationally focuses on low cost. It is entirely reasonable for the government to buy the cheapest toilet paper, but that reasoning is detrimental when applied to IT products and services. Under the FAR, most proposal teams have learned to low-ball the bid: the lowest bid wins the contract. Every IT consulting firm, large and small, rewards the proposal teams that win contracts. But the trouble starts, when the technologists have to perform work under the contract that was priced impractically low. At the moment, there are no remedies to this situation. If you find yourself in the midst of such a low-ball contract, your only recourse is to keep ploughing away or to quit. Such is the sad state of affairs in IT government contracts. The only recourse is to refresh the FAR to reflect the realities of the modern IT world. As with anything political, that is easier said than done.
Modern large applications collect and hold users’ private information, such as financial positions, purchasing patterns, social networks, health records, political persuasions, contact information, etc. Sometimes, these systems continuously track all online activities of the users. Over the past two decades, IT companies have profited billions of dollars by trading in the users’ private information, and the law continues to permit them to do as they wish. In a perfect world, corporate law would charge IT organisations and their employees with fiduciary duty toward their users. That is not the case, however, at present.
Trading in generalised trends is a legitimate practice; its origin may well reach back several millennia. But trading in particularised data that can be traced to an individual is a clear violation of that individual’s privacy rights. Our Constitution prohibits government entities from partaking in privacy trade, but the law has very little to say about private entities trading and profiting from their customers’ private data. And under corporate law, the corporation and its directors must do their utmost to extract profits by any lawful means. So, despite the stench of unethical conduct, corporations continue to trade in private information.
So, for the time being, the best that the rank and file technical staff could do is to sniff out the stench of privacy violation in their daily work, record the occurrences, and report them up the chain of command. Make the top executives aware, when they are making unethical, albeit legal, decisions to violate the users’ privacy. After all, the responsibility to balance the myriad concerns inheres in the executives.
Another important issue is customer billing. The days of companies selling their software as products is gone; software is now a billable service. Many billing systems employ a simple, flat-fee model. But some billing models are quite complex, based on intricate formulae involving usages of computation time, storage space, network bandwidth, and many other resources. Complexities inevitably lead to errors in calculations. Sometimes, users end up getting charged for things they did not use and, other times, the users are under billed and the company ends up eating the loss. Billing mistakes like these are a serious type of error. After all, the company’s reason for existence is to make profits and it is the billing system that is making all the profits.
If billing errors are discovered, they can be fixed. But the trouble ensues, when various subsystems are owned by different teams and the errors are smeared across team boundaries. The instinct to protect one’s turf always trounces the desire to aid others in the organisation. So, the stench of politics that permeate the office is carried by the breeze of commerce to all the customers. Even if turf wars are the norm in an organisation, teams should call a temporary truce, whenever the customers’ funds are at issue.
All corporate actions are governed by law.
A corporation is a legal person, an entity created by the operation of state law. Its primary duty is to make profits for its shareholders. There are many laws governing the conduct of the corporation. But being a morally upright, environmentally conscious citizen of the world, is not a requirement for a corporation. Nevertheless, you, an employee of the corporation, is no mere legal person, but an actual person. As such, you have moral and ethical duties. Above all, you must also abide by the laws that govern the corporation.
Employees owe the employer duties of obedience, loyalty, and due care. Obedience, in the employment context, means the employee must follow the superior’s lawful orders and produce lawful work products for which he is paid a salary. Loyalty means the employee must not betray the employer’s trust in him, he must not appropriate his employer’s property, and he must not disclose his employer’s confidential information, such as trade secrets, market analysis, customer list, etc. And due care means he must perform his work with care befitting his office, and in performing his duties he must not expose his employer to undue risks. In return, the employer woes the employees wages, workplace safety, non-hostile work environment, work-related indemnity, and due process.
A typical small business owner mistakenly believes that because he owns the company, he owns his companies’ assets, too. In the eyes of the law, the founder and the company are two separate legal entities, with separate assets and liabilities. So, the owner’s converting the company’s cash, office space, furniture, vehicles, and the like for his personal use constitutes misappropriation, an illegal conduct. And it is important to note that the company owner who draws salary from the business is an employee, so he owes his company certain duties just like other employees. It is not uncommon for employees to raid the office supplies closet for pencils, staples, batteries, etc., and putting the supplies to their personal uses. Although this is technically misappropriation, the law does not concern itself with trifles. Regardless, even such de minimis misappropriation is still inappropriate.
In general, a professional STEMer’s work product is an intellectual property owned by the employer. So, if you wrote a piece of code at work, although the copyright therein attaches to you nominally, the code, as your work product, belongs to your employer. Were you to copy that code and use it in your own project, you are misappropriating that code. But if you recreate a similar piece of code from memory, that new code is yours. But if you are an independent consultant hired by the company to perform a specific task, but without instructing you on how to perform the work, the work product belongs to the company but the tools you created to help you perform that work belongs to you. These are the finicky bits of intellectual property law.
Doctors, lawyers, and accountants must comply with stringent regulations and professional codes of conduct that prescribe the proper handling of sensitive information, but IT practitioners are not subject to regulations that specifically target them. Nevertheless, technologists who work with systems that hold finance, billing, medical records, and other sensitive information should exercise extraordinary care when handling such sensitive information, and must comply with the regulations that govern the overarching practice areas. You have heightened duties, when you control other people’s private information. In particular, those technologists who have access to company accounting system, personnel system, authentication system, and the like must never misappropriate the information they may chance upon in the course of performing their technical work. In short, do not spy on your coworkers.
Countless technology companies have fallen prey to the temptation to sell vapourware, and many more will follow. It is a widespread practice in the IT industry. Indeed, when a consultancy agrees to build a custom-made device for a client, that product is technically a vapourware, at the time the agreement is made. Selling vapourware, per se, is not unlawful; the law tolerates commerce-related puffery. Of course, the company must eventually deliver the product, even if over budget and beyond deadline. If the company failed to deliver, yet the executives hoodwinked the investors and perpetuated false promises, that is criminal fraud. In general, the top business managers who possess no technical background sell the vapourware, and the technologists who are on the hook to turn the concept into product. So, if you are a senior technologist within the company, you must do your best to convince the business managers to avoid making unrealistic promises to investors and customers. This, however, is a tall order.
The corporation and its employees owe a number of duties to the customers, too. Today, one of the most important duty owed to customers is to keep their private information safe. But most technology companies, have breached this duty. Many companies sell their customers’ private information, without the knowledge of the customers. Many other companies failed to safeguard their customers’ private information from online theft. So far, the law has been lagging far behind, and there are scant few protections and remedies available to customers.
For service companies—power companies, communications companies, etc.—their duty to deliver services to the customers is governed by the service level agreement. Large public utilities must also comply with several regulations that govern service delivery, such as making services available to underserved regions, being emergency services available for essentially 100% of the time, and so on. For example, if a customer subscribed to 100 Gb/s broadband service, the company must ensure the service operates at or near the stated bandwidth, the service is available for 99.99% of the time, and so on.
Service life is a problematic are for many technology companies. Say, a rich customer paid a small technology firm to implement a bleeding-edge home security system. In a few short years, the technology will become outdated. Depending on the service agreement, the company may owe no duty to upgrade the system in perpetuity. Years later, many components will begin to fail. When the home owner asks the company to repair the system, there may no longer be any compatible components available on the market. Indeed, the company may well be out of business. There are a few things the customer can negotiate, in order to protect his investment. If he has a reason to doubt the company’s longevity, he may ask the design schematics and other artefacts to be deposited with a trustee, so that if the company were to become defunct in the future, he would at least be able to hire another company to make necessary upgrades and repairs. Sophisticated customers are, of course, well aware that obsolescence is part and parcel of technology.
The corporation also owes certain duties to other companies with whom it is engaged in business agreements, such as intellectual property licensing agreement or partnership agreement. If a company licenses another company’s technology, the licensee must protect the licensor’s trade secrets and must not misappropriate the licensor’s intellectual property.
Today, technology has become much too complex for one company, even an Internet giant, to design, test, manufacture, and sell all its products. For example, Apple designs the iPhone, but the phones are made in China, under a partnership agreement with a manufacturer. Without such partnership agreements, there will be no iPhones, iPads, or any high-tech device. Under this manufacturing agreement, the Chinese manufacturer is prohibited from selling the devices as unbranded phones in the local market.
One of the largest consumers of technology goods and services is the US federal government. The government makes purchases under the Federal Acquisition Regulation (FAR). It makes economical sense for the government to contract with a specialist firm that makes fighter jets, instead of establishing an aircraft design bureau of its own, since aircraft design and manufacture is not a government function. The FAR requires the government to solicit offers from multiple manufacturers, and to choose the one best deal. Today, IT services, too, are purchased under the FAR.
As all things governmental, IT service contracts performed under the FAR are rife with inefficiencies brought forth by unnecessary bureaucracies. Often, technologists working on government contract IT programmes find themselves buried under regulations, rules, policies, and guidelines. This type of inefficiencies protect the government officers and contract consultants from legal scrutiny, but the government’s IT systems are left to the mercy of well-funded, sophisticated hacker groups operated by foreign adversaries. For the foreseeable future, there is no remedy to this situation. But as a consultant technologist, you should be cognisant that although you are paid to follow the procedures and to protect your firm against legal risks, your real duty is to keep the government’s infrastructure safe.
Only those who love the work last.
To last a long time in a career, you must be professional and you must continuously improve yourself. And most importantly, you must love what you do. Otherwise, you will burn out and become a bitter, unprofessional grump, whom no teammate can countenance.
In IT today, if you love your work, you are in the fortunate minority. Loving what you do is the most important requirement of a long, fulfilling career. Another requirement is to be professional: be a team player, but do not engage in inter-departmental politics; be courteous and generous; go above and beyond your duties, but prioritise your tasks; take your work seriously, but not yourself; be truthful when offering an opinion or an advice, and be humble when receiving a critique; always be mindful of your mental health and that of your teammates; never stop learning, but do not be a dilettante. It is all common sense stuff, really.
An important aspect of professional conduct is making yourself available when your team needs you. That means you, as a salaried employee, may occasionally have to work unpaid: sometimes, staying at the office late into the night alongside your teammates; sometimes, answering the phone during your holiday at the beach. Most of the time, your manager will allow you to balance out or unpaid overtime hours against your normal work hours. But when the team is working on a tight schedule, you may have to delay recouping your unpaid hours and, in rare cases, you might even give them up, for the good of the team.
We work with others in a team and in an organisation, because we cannot do what needs to be done, individually. As a collective, we sometimes have to make little sacrifices. Usually, these small detriments are spread out evenly across time and space, and are tolerable. But if it becomes intolerable, either because your employer is exploiting your good nature, or because the work intensity never dissipates, you must take appropriate actions to restore balance: raising the issue to your team and your manager, offering ideas to improve the conditions, etc.
In IT, things are changing continuously. So, we all have to pull a little harder, from time to time. But chronic shortage of time and resources is a symptom of poor management. This is why I have always maintained that technologists should be managed by a senior technologist. If nothing you do diminishes the management’s unprofessional conduct, then you have but one recourse left: move to a better place.
If you last a long time in an IT career, you are guaranteed to witness many changes over the years, in technology, in market, in customers, in colleagues. So, it is important that you improve yourself, continuously. By that, I do not mean just keeping up with fashionable trends or reacting to them. I mean to anticipate substantive trends and to be an agent of meaningful change within your organisation. Inevitably, to be an agent of change, you must, yourself, change, too.
A substantive improvement is not merely replacing an older thing with a slightly newer thing of the same kind, say like a JavaScript programmer switching to TypeScript or a manager getting yet another project management certificate. To improve yourself continuously, you must constantly step beyond your comfort zone. That is, keep learning new and different things with which you are unfamiliar, but things that are connected to your area of speciality.
Learning for self improvement comprises both theoretical knowledge and practical skills. An improvement could be as monumental as getting a PhD or as minuscule as studying a new algorithm. The key is to keep improving, incrementally, persistently.
As a learner, you must accept that not everything you learn will have a direct and immediate impact on your career, and that some trifle thing you picked up long ago can one day give you a spark of inspiration you need to accomplish a major goal. As a rule of thumb, you should at least have a working knowledge about something, if it is related to your work. Just as an engineer must possess a basic understanding of the relevant regulations that govern his work, an IT executive must possess a basic understanding of the technologies being used and produced by his staff.
Self improvement is necessarily about the expansion of the mind. So, it is imperative that you avoid narrow mindedness. Keep an open mind. Seek out areas in which you could improve. Look for things that you could learn from others.
Being open minded is not the same as blithely chasing every new trend, however. As an IT practitioner, you are responsible for bringing new technological developments to your managers, so you must monitor the industry trends. But if you keep fluttering from one new thing to another, you will only waste time and resources without accomplishing anything. So, keep an eye on trends, by all means; but always pass your enthusiasm through the filter of judgement.
Realise, also, that 10-minute YouTube instructional videos can never make you an expert in anything, no matter how many of them you have watched.
The old survivor has the obligation to mentor the young.
Mentoring is a subtle craft that can be practised only by those with deep reservoirs of knowledge, skill, experience and, above all, generosity and patience. That is, not every in a senior-level role can be an effective mentor for the juniors. Indeed, a senior person with poor mentorship craft can damage the careers of many junior team members.
Some mentors are indelicate; they are prone to chest pounding or stomping down upon the mentees. It is clear that you, the mentor, are the most senior staff on the team, so there is no need for you to make a show of your prowess. Mentoring is not a competition of skills; it is a transfer thereof.
As an independent consultant, I have observed on several occasions genuine Machiavellian moves among the in-house staff. An astute mentee bides his time and soaks up as much as he could while working under a generous mentor who is a long-time employee of the company. One day, when the mentee feels secure enough, he launches a coup d’état to displace his mentor. The senior executives see only the savings to be extracted from replacing an older, more expensive employee with a newer, cheaper one, who appears to possess the requisite skills.
One of the most difficult tasks of a mentor in IT is actually non-technical in nature—getting through to people. As a mentor, you already have the podium, as it were. So, most junior teammates will instinctively look to you for guidance. But there will always be a few mid-level or senior staff who will resist and subvert this natural order. Some reject your guidance out of habit, some out of arrogance, some out of laziness, and others out of their political allegiances.
There are no rules of thumb on how to penetrate recalcitrance; every situation is unique. So, you must learn this skill on the job. You do, however, have many techniques at your disposal. You can reason with the person to draw out his inner collegiality. You can use the peer pressure of a good team to breach his barriers. You can bring to bear your authority as the team leader, but you must wield your authority with subtlety, lest you worsen the already-difficult situation. You can escalate the unruly staff to senior management or the human resource department. And you always have the annual performance review.
In general, however, laziness is an unsolvable problem and politics is an intractable problem. Your only option in such cases is to remove the person from the team, thus protecting the team from negative influences. Sloth is not tolerated in a high-pressure industry like IT. But a Machiavellian man may blend better in a politically charged environment, elsewhere.
In may ways, the mentor is a substitute college professor for his more junior charges. He is responsible for teaching and training them, and to give them the knowledge and the skills they will need in their future careers. Just like when teaching a college course, you must establish a curriculum, schedule it and, at regular intervals, evaluate your mentees’ performance, culminating in the year-end employee evaluation. Then you repeat, next year. Mentoring, like teaching, is incremental and cumulative. The mentor, like the professor, must be continuously invested in the progress of the mentees. There is no hands-off style of mentorship.
For you to be able to evaluate your mentees’ progress at set intervals, you must set realistic, but challenging, goals. And unlike a college professor who teaches from one curriculum, you must develop individualised curricula for each mentee, based on their varied backgrounds, experiences, and career goals. And each mentee has short- and long-term goals. That means you should establish goals in collaboration with the mentee.
A common long-term goal of junior staff is to earn a graduate degree, either in STEM or in business, and rarely in law. Your involvement in those situation is minimal. You need only to help manage the mentee’s workload, so as to prevent undue stress. Many companies reimburse a set amount for each course the mentee takes.
But setting short-terms goals are more intricate. For a junior technical staff, say a first-year programmer, you may set a modest goal, such as requiring him to pick up a programming language that he never studied in college—something simple like Standard ML or OCaml will be of immense value, because most new programming languages have adopted the functional features of these languages and also because typical computer science curricula no longer emphasise functional programming languages. For a mid-level programmer, he may aspire to rise to become a system designer, or to lead a cadre of programmers specialising in one aspect of a large system. For database administrators, system administrators, and network administrators, you may progressively enlarge their responsibilities by shifting them from administering small, internal systems to leading the administration team in charge of large, public-facing production systems. Also, do not neglect that all administrators must be proficient in shell scripting and at least one of the so-called easy-to-learn languages: Python, Lua, Go, Nim, etc.
The important thing for the mentor to do is to set realistic, challenging goals, and to hold the mentees accountable for their attaining the goals. Also, the mentor should inculcate in the mentees the philosophy of lifelong learning: learning continues even after retirement and familiarity with topics outside the concentration is always beneficial.
And remember this: as you mentor others, someone else is mentoring you. You must not neglect your own educational and professional advancements.
A leader is a visionary, a nurturer, a mediator, and a negotiator.
A technical team leader or a technical manager must, by definition, possess the requisite educational qualifications and professional experiences. But the leader must also have the right mentality: collegiality, forbearance, determination, assiduity, and all the rest.
The team leader defines the team’s short-term goals based on the senior management’s vision of how to accomplish the organisation’s mission. By “vision”, I do not mean broad, aspirational proclamations that the top executives are wont to make, for such vague statements are not actionable at the team level. The team leader must, therefore, translate the executive vision into team action—a rather tall order, for sure.
In IT, it is generally impossible to contemplate long-term goals, at the team level, because the one constant of this industry is change—change driven by the fluctuations in the needs of market, competitors, users, and executives. This instability places additional demands upon the team leader, obliging him to monitor the trends and to position his team accordingly. Indeed, XP, Agile, and other iterative methods were developed to cope with this long-term unpredictability.
The team leader’s primary role is to empower and to enable the team to accomplish its assignments. The leader must ensure that the staff has the necessary resources, and especially time, to complete the work with distinction. The leader must monitor and balance the workloads of the individual team members. And the leader must shield the team from external distractions and, on occasions, detractions. One of the most important aspects of the leader’s role is that he is a team member, as well. This is often overlooked, however.
There are several categories of ineffective IT leaders: blamers, abdicaters, sellers, talkers, soloists, and teamsters. Blamers are like mushrooms in the IT industry. In an ideal world, praise travels downstream, blame upstream. But the reverse is much more common in IT, these days. Under such leaders, the mental health of the staff and their productivity suffer. Abdicaters are the ones who delegate away all their leadership duties to the subordinates and show no interest in the affairs of their teams. This behaviour is quite common in IT, especially in government consulting world. Often, abdicaters are also great blamers. Sellers are those leaders who believe their only role is to promote their teams’ virtues. Their focus is fixed entirely on things outside the team. They are detached from the team. So, they do not know their teams’ strengths and weaknesses. In the process of making a sale, they invariably overcommit their teams. Talkers are the chief distractors of the staff. They do not possess hands-on experience. But they have cursory interests and superficial skills in every new piece of technology. Instead of doing the work, they talk of doing the work. Under such leaders, teams are constantly being peppered with irrelevant questions and inappropriate directions. Soloists are the skilled, but defeated, leaders. They have given up leading dysfunctional teams, and have decided to do all the work by themselves or with the assistance of one or two team members. Under such leaders, teams may well complete the assigned tasks, but they do not grow as a collective. Teamsters are the drivers of animals. They treat the staff like a herd of cattle to be driven with whips. Under such leaders, the team members perform the work out of fear, but at the cost of their happiness and satisfaction for job well done.
As the team leader, you must also pay attention to the non-technical aspects of the work: ethics, legality, and politics. You must ensure that the work you and your team performs is within the bounds of ethics and, most certainly, in compliance with applicable rules, regulations, and laws. You can acquire training in ethics and law in classroom settings, but there are no comparable learning opportunities for corporate politics; that you must learn on the job. You, the team leader, must do your utmost to quash intra-team politics and to keep external politics at bay.
In heavily regulated sectors, such as aerospace, nuclear power, medicine, etc., IT work, too, is subject to the governing regulations. For new entrants to IT and émigré from unregulated sectors, like social media and web development, having to work under strict procedures may feel oppressive. It is your duty, as the team leader, to explain to the staff the importance of abiding by such procedures in safety-critical industries. And you must also monitor their compliance.
Sometimes, meaningless procedures evolve over time and come to dominate IT work. Such a scenario is especially common in government consulting. In this environment, new regulations always give birth to new procedures. That is the nature of the business. But what is problematic is that the IT procedures are written by technologists who cannot read regulatory texts. As such, the procedure either miss the mark or are overly burdensome. You must, therefore, take the initiative to analyse the responsiveness of the procedures to the commands of the regulation. That means you must know how to interpret regulations, or know someone who can. In a large organisation, you can seek advice from the corporate legal department, if necessary.
A technical leader usually partake in interviewing candidates, especially those being considered for senior-level roles. Sometimes, you may wish to ask technical questions. But your role as the leader, is to keep the interviewing process collegial and real—to give the candidate the opportunity to assess your team’s culture and style, and vice versa. In any event, you should eradicate from the interview process those childish, time-wasting questions, like asking the candidate to list five best this, or ten worst that—things anyone can look up online. The primary purpose of the interview process is to ascertain the candidates’ ability to work effectively within your team.
Every technical team leader is a technical manager. So, you must manage people, projects, and products of the team. You need to have the experience of establishing and overseeing the whole process. Although most people in IT equate the word “process” with the term “Agile”, very few teams actually follow any substantive process; most simply “wing it”. In safety-critical specialities of IT, however, process always involves documented procedures and requirement for documentation.
The technical team leader is the mediator. He mediates between teammates with opposing interests. He mediates between the team and the managers. He mediates between the team and the users. The mediator impartially balances the interests of the opposing parties, so that everyone feels the mediated settlement was fair.
The technical team leader is also the negotiator. He negotiates on behalf of his staff for pay rises and other benefits. He negotiates with vendors to secure his team and employer get the most advantageous deals on products and services. He negotiates with users to ensure that his team can perform the work with the most support and the least stress.
In general, employees want higher salaries, benefits, and resources, but the employer wants to minimise these expenses. So, the team leader, who negotiates on behalf of his team, often finds himself playing the zero-sum game with his line manager. This can be exasperating when his team feels slighted because they perceive that the management allots resources in favour of other teams. The team leader’s mediation and negotiating skills are put to the test, when the resources are scarce.
Sometimes, you, as the team leader, may find that your interests are not fully aligned with those of your team. Say, you inherited someone else’s established, tight-knit team, and the team resents you for displacing their beloved former leader. This may seem like a quandary, but for a competent leader, it is not. Think of it this way: you were appointed the leader because the management has faith in your experiences and skills, and the team you inherited is already a tight-knit team that respects its existing hierarchy. All you need do, then, is to establish a good rapport with the senior members of the team who command the respect of the staff, and extend the reach of that goodwill therefrom. Clearly, you need the support of these senior members. And they have a vested interest in maintaining coherence and continuity. So, you and the senior staff may end up in an informal negotiation session. Just recognise that this is neither a mediation nor a negotiation. You are an opposing party, not an impartial mediator. Moreover, you, as the appointed leader, hold the power advantage in negotiations, and if you exploit this advantage, you will lose support of the senior staff, thereby destroying your chances of success with the team. Exercise due care.
And sometimes, if you are particularly unlucky, you inherit a dysfunctional team. Though a more difficult prospect, this, too, is not an impossible situation for a good leader. Those who join a company, for the most part, are not sociopaths. People behave in unprofessional, anti-social ways, only because there is no team spirit, no guidance, no vision, no leadership. A good leader who inherits a bad team always works hard to find remedies. The first thing that you, the new leader, must do is to establish your authority over the unruly, irresolute team. You can do this by replacing the ineffective senior members with effective ones whose abilities you know and trust. Often, you may have to roll up your sleeves and do the work, yourself. This is why I maintain that technical leaders must be technologists, in the first place. The next thing you must do is clearly and succinctly to communicate your mission, vision, and expectations to the entire team. This is analogous to a ship captain who announces the destination well ahead of departure time, so that passengers who got on the wrong ship may disembark, before the departure. And perhaps the most important task for you is to establish effective procedures within the team for oversight and communication. The lack of oversight and communication are almost always the causes of dysfunction. When the team leader communicates with the team and the team members communicate with one another, everyone knows what to do, both individually and collectively. When each subunit within the team is overseen effectively, the team leader needs only to oversee the overseers.
In any case, it is vital that you eradicate insubordinate conduct. A team is a hierarchy, much like its parent organisation and the society at large. There is no place for the unruly in a hierarchy. Most people view a hierarchical organisation chart as a pyramid: a pointy leader valiantly perched atop a wide base of subordinates who look up approvingly at him. But in truth, a good leader holds up the team—somewhat precariously at times—upon his shoulders, so as to empower it to achieve its full potential as a collective. When the weight of the whole upside-down pyramid is upon your shoulders, you can ill afford to keep a misaligned stone within that smooth structure, lest the whole thing crumbles upon your head.
In most jurisdictions, employers hire whomever they please and employees work for whichever firm they choose; employees resign at will and employers dismiss at will. But this so-called “at will” is more theory than practice at large organisations. Large organisations are particularly prone to law suits brought about by dismissed employees under anti-discrimination act, disabilities act, civil rights act, etc. So, large companies take pains to document insubordinate conduct over weeks and months. Even when there is documented evidence, many people and their unscrupulous attorneys try their luck at nuisance suits, because it is often less costly for a large company to settle law suits than to try them in court. So, before you decide to sack an insubordinate staff, consult your human resources department and the legal department.
The situation where the insubordinates roam free is federal government IT service programmes. Everyone knows that some federal agencies are run like reality shows on the tele. In this putrid political pit, unprofessional consultants make end-runs round their programme manager by striking up friendships with government employees. At the best case, this is a conduct unbecoming of a government officer. But in the worst case, this is the government’s improper interference with the consulting firm’s management of the programme. If you find yourself in that unenviable position, your first task must be to establish a communications protocol that respects the chain of command. Only the senior staff on the programme may communicate directly with government officers. This may seem inefficient but, in reality, it is the best possible strategy. This is not putting a restraint on the information flow, but it is making the information flow manageable for the benefit of all parties. It is all too common for critical information to get lost in undisciplined back-and-forth between the government and the programme staff, exposing the government, the consulting firm, the programme manager, and the programme itself to undue risk.
In a normal course of contract performance, you, the programme manager, act as the mediator between the government officers and the programme staff, ensuring fairness to both parties. But on occasions when the government officers and the programme staff disagree, the government contract officer and you become chief negotiators for the opposing parties.
As a senior leader within the organisation, you may sometimes have to participate in legal negotiations: supply contracts with vendors, partnership agreements with potential corporate partners, licensing agreements with intellectual property owners, and so on. If your organisation is represented by a corporate lawyer, then you serve as his technical advisor during negotiations. But if you are representing the organisation in your capacity as a senior leader, then you must learn the art of negotiations.
salary negotiations with candidates and departing staff, contract negotiations with potential clients, partnership agreement negotiations, licensing agreement negotiations
Every experienced STEMer who has lasted a couple of decades in the industry knows about the topics I discussed, here. But no new entrant who enters the industry came equipped with a career plan, since no college curriculum covers such topics. I hope the information I presented above serves as a guide to the careers of young STEMers, especially those in the IT industry.
In sum, I repeat these admonishments: