Using command line tools proactively

I still remember the day that I firstly saw the books like "the Linux command manual". Most of them are just (or similar to) the collection of the contents from the "man" command output. I was surprised at "How a person could memorize or know so many commands and their usage." There are hundreds of commands, and thousands of their corresponding options and usages.

Many people, at least myself, figure out which command we should know and how to use it by googling a specific question. For example, google "How do I list files in a directory with Ubuntu Linux", and pick up (randomly) the first few searching results to follow. As the time goes by, I know more and more commands until I could handle most of the issues in my life.

Then the learning curve reach a plateau. I won't learn new command or new option of a known command for longer and longer period.

"How the people on the Stack Overflow know so many variant ways or commands to handle a similar problem?" I seldom (and can't most of the time) walkthrough the manual of a command, and I suppose many people don't as well. For example, the "dd" command has a lot of fancy and useful options and features to use, but the default value should work like a charm in more than 80% of life problems. If I don't know there is such a option or an extra feature to use, how am I aware that I could use them?

Besides from randomly googling and waiting for someone's answer on Stack Overflow, there is another way to think outside the box: think proactively.

To think proactively here means the following points of view have been considered in our mind:

• What problem I am going to resolve?
• What's the essential property of this problem? Does this property has similar problems as well?
• What feature this tool should has to resolve this problem essentially?
• What kind of the tool to resolve this problem may be? How this kind of tool to resolve the problem?
• What is the result after I applied the tool to the problem?

Let's take the "dd" command as an example again. Assume our problem is "to clone one disk". Then the essential property of this problem could be:

• How to clone this disk faster? - Is there any option to make it faster?
• How to clone the disk reliably? - Is there any option for me to check the status frequently?
• It is an I/O problem - There are very likely to be input and output related features.
• It is an operation on block device - I have to think form the block device point of view.

And then it could be:

• Speed: what is the potential features to make the read/wirte IO of block devices faster? - read/wirte chunk size. Error handling.
• Reliability - Is there any process status reporter? Is there any error handler?

So I might figure out the "bs" option may play a role for the speed, and there are "sync" and "noerrors". I could suppose there should be a option for progress. It is status=progress then.

This mind is similar to the mind when trying to find out an solution to a problem, answer to a question, or debug code. Essentially they are just to figure out the goal, collect the associated 

information, apply it and review the result consciously.

For example, I am working on SGE (Sun Grid Engine) infrastructure recently. I was prototyping a solution to build the infrastructure automatically with LXD/LXC. When I complete the prototype I move the solution Juju/Charms to MaaS. I was blocked then. However I could soon find the root cause out by thinking proactively, like:

• The error looks like a network issue and permission issue.
• To build a SGE scheduler is a question about communication between nodes.
• When I setup the prototype successfully, which part relates communication/networking/permission.
• Is the same step applied to the new infrastructure flow?

Then I could understand what kind of commands (qconf for example) and the associated options I may look into. : )

In conclusion, when you have some background knowledge already, try to think "the tool that I has known would be great if it could has this feature. Does it have this feature?" rather than "google the problem directly." To google a problem on the internet you could most of the time just get an entry level answer, none, or noise.