Many researchers have studied, experimented, and investigated ways to enhance the heat transfer in helical tube exchangers. The reason is that helical coiled exchangers are using in a wide range of industrial operation all over the world because of its great heat transfer efficiency and compact size and the fluid flowing through the helical tube has additional benefits over straight tubes. This design of exchanger can benefit from nanofluids, which provides the excellent thermal performance. In this work, a helically coiled exchanger is using to investigate different nanofluids effects on the characteristics of heat transfer to improve the coefficients of heat transfer as well as the experimental model’s net heat transfer, i.e., based on numerical simulations. Throughout this study, there are five different nanofluids are used in the shell and tube sides respectively i.e., CuO/water, Al2O3/water, SiO2/water, TiO2/water, and ZnO/water, with volume fractions ranging from 1% to 5%, while the shell and tube’s mass flow rates are kept constant at 0.019 kg/s and 0.016 kg/s, respectively. So, we have looked for correlations for Al2O3/water, SiO2/water, CuO/water, ZnO/water, and TiO2/water nanofluid characteristics in research journals and created the model in ANSYS WORKBENCH 2021, afterwards meshing it and solving it in ANSYS Fluent solver for various nanofluid concentrations. The base fluid is water in both the situations, with the hot fluid that flows through helical coil and the cold fluid flows through shell in a counterflow manner, using the tube material i.e., Copper. The physics monitoring equations such as mass momentum and energy are solved utilising viscous laminar model because of its Reynolds number. The temperature results are validated for the experimental results that are referred to, and the heat transfer values are plotted. The results of using above mentioned different nanoparticles shows that when using higher particle volume concentration nanofluids on the side of tube, the coefficient of heat transfer and the net heat transfer found to be high. When using higher particle volume concentration nanofluids on the side of shell, however, the coefficient of heat transfer coefficient and the net heat transfer found to be low. Among the five nanofluids, the overall coefficient of heat transfer and the net heat transfer increased the most in Al2O3/water nanofluids, while the coefficient of heat transfer and the net heat transfer found to be the lowest in SiO2/water nanofluids. According to the findings, for helically coiled heat exchangers, Al2O3/water nanofluids have the highest efficiency of heat transfer.