3. Parallel Computing -

3.1 What is Parallel Computing ?

In the process of Parallel Computing, large problems are broken down into smaller problem. These problems are independent, similar parts which can be executed by multiple processors, simultaneously. The processors communicate through shared memory. Towards the completion, these results are combined to the complete output. Parallel computing mainly aims to increase the available computation power for faster problem solving and application processing.

Figure 2. Parallelism for a multiple entity problem.

(Source: https://hpc.llnl.gov/training/tutorials/introduction-parallel-computing-tutorial)

Figure 3. Parallelism for a singular entity Problem.

(Source: https://hpc.llnl.gov/training/tutorials/introduction-parallel-computing-tutorial)

3.2 Advantages of Parallel Computing-

• Time Efficiency- saves time, allowing the execution of applications in a shorter time.

• Fragmentation- since a larger problem is broken down into small fragments, larger problems are solved in a short point of time.

• Parallel Computing is better used in modelling, simulating and to understand real-time applications.

• Cost Saving- it can be built using cheap-commodity components. Also the task completion time is shortened which potentially reduces the cost.

• Suitable for Complex Problems- when given limited computer memory is provided, parallel computing is the most suited method.

• Organization & Storage- the fragments can be kept more organized and parallel computing has huge data storage and quick computations.

3.3 Disadvantages of Parallel Computing-

• The programming required for parallel computing is a little difficult, but on through study it can be accomplished

• Although parallel computing helps solve complex and large problem, but it can have some adverse effects on control algorithm

• Even though parallel computing is cost effective, but some extra cost can be incurred due to synchronization, thread creation/ destruction, communication, data transfer, and many more. Sometimes these costs can also exceed the profit that one has earned due to parallelization.

• To make it efficient, advanced cooling technologies have to be used incase of clusters.

• Since, multiple processors are used in parallel computing, the power consumption is also high.

• Debugging can be a complicated and tedious process when it comes to parallel computing.

3.4 Applications Of Parallel Computing-

Following are the applications of Parallel Computing:

1. Computational astrophysics

2. Geoprocessing (or seismic surveying)

3. Climate modeling

4. Agriculture estimates

5. Financial risk management

6. Video color correction

7. Computational fluid dynamics

8. Medical imaging

9. Drug discovery.

4. Distributed System-

4.1 What is Distributed Computing ?

In Distributed computing the software system is distributed among multiple computers. Despite of the software components being distributed into multiple computers, they run as one system. This type of computing improves efficiency and performance of the system.

On a brief note, distributed computing is limited to programs which have shared components within a limited geographical area. But it includes task sharing. This type of computing requires lot of soft skills and tooling.

Figure 4: Distributed System

(Source: https://www.sciencedirect.com/topics/computer-science/distributed-computing)

4.2 Advantages of Distributed Computing-

• Data sharing- sharing of data from component to component is easy since in a distributed system nodes are interconnected for easy communication.

• Scalability- basically the extent and size of the system can be changed for a particular node or a system. Also multiple nodes can be added in a particular node for a distributed system.

• Failure handling- Since the system is not based on a single the failures in any node can be resolved individually without disturbing other nodes in the system.

• Reliability- the efficiency of error handling makes distributed systems more reliable.

• Efficiency- involvement of multiple computers saves time making the job highly efficient.

• Lesser delay- distributed system provides with a lesser delay rate. Since the nodes are distributed and tasks can be distributed as well.

4.3 Disadvantages of Distributed Computing-

• Security issue- due to multiple nodes and an open setup being in action, leakage of data from a node can be a concern but with right security options, this can be avoided.

• High set-up cost- it is a little expensive setup, since a distributed system has many software and hardware components and their respective maintenance costs add up to the expenses.

• Data loss- data can be lost during transmission from one node to the other in the way from source node to the destination node. This can be a concern, since important and private data can be lost.

• Difficult to handle- the distributed system is made up of multiple software and hardware components which have to maintained form time-to-time for smooth operation. Due to the high complexity the software components should be paid special attention for error free computing

• Overloading issue- since the data has to be passed from source node to the destination node, every intermediate node has to contribute with the data tasks that have been designated. But if a situation arises where all/ many nodes try to sent information at the same time this can cause network congestion or overloading.

4.4 Application Of Distributed Computing-

Following are the applications of Distributed Computing:

1. E-commerce platforms like Amazon, Flipkart, etc.

2. E-learning platforms

3. Monitoring Systems

4. Automation Processes

5. Online Banking Systems

6. Online Gaming, etc.

5. Concurrent System-

5.1 What is Concurrent Computing-

Concurrent computing is a process under which multiple programs are run by only one computing system by the process known as alternating timeshared slices. Basically time-sharing is a method of operation. by using this method multiple users with different programming tasks interact simultaneously with a single system.

5.2 Advantages of Concurrent Computing-

• Computers processors would be used to full potential- since the different cores of the Central Processing Unit (CPU) will be performing different tasks

• Faster response speed- since one system will be multitasking, this will highly reduce the processing time.

• Easier to develop- concurrent programming is considered easy to develop since it uses multi-threading

5.3 Disadvantages of Concurrent Computing-

• Increase context switching- affecting execution speed- if the task is not completed within a given time

• Deadlock- the deadlock is a state of the system, when the system is in a waiting state since the required resource is being used by another user

• Reduced Efficiency- since the software and hardware resources get distributed within the users, this causes resource limitation.

5.4 Applications of Concurrent Computing-

Following are the major applications of parallel computing:

1. The most important application of concurrent computing is in simulation and controlling of discrete systems.

2. Concurrent computing can also be used in addition to parallel programming to increase the efficiency.