The most valuable part of machine learning is building and effective "Predictive model". This is the development of models that get trained on prior data and then make predictions on new (unseen) data. Here is my general rule of thumb process when working with both datasets and algorithms:

1. Pick one group:

(a) Data

(b) Algorithms

(c) Parameter Tuning

2. Pick one method from the group.

3. Pick one thing to try of the chosen method.

4. Compare the results, keep if there is an improvement.

5. Repeat the steps again ...

1. Improve The Performance With Data

I - Getting more data

II - Selecting reasonable predictors

III - Preparing the data

IV - Resample and transform the Data

V - Consider performing feature selection

2. Improve The Performance With Algorithms

I - Evaluating the learning process with a reliable, e.g., k-fold cross-validation

II - Selecting the reasonable "Evaluation Metric"

III - Try several machine learning algorithms (i.e., don't select the algorithm blindly)

IV - Select the algorithm that produces best result

3. Improve Performance With Parameter Tuning

I - Optimizing one parameter using, e.g., "Cross Validation" parameter selection technique

II - Optimizing two parameters using, for instance, "Grid Search" method

III - Optimizing arbitrary number of parameters using, for example, "Multi Search" approach