Here are some points to keep in mind when you ask us to develop scraping

■Installing Selenium

The scraping program that Uesugi will create will be in the form of a browser controlled from VBA using a framework called Selenium.

Therefore, you will need to install Selenium before using the program.

How to install Selenium

■Instability of scraping

As noted above,

VBA→Selenium→Browser→Target website

As described above, the VBA program executes the process in the form of "VBA→Selenium→browser→target site," which makes it somewhat more unstable than a normal VBA program.

In particular, since the browser is used to connect to the target site, communication with the site's web server occurs, just as in normal use, so there may be various unstable factors such as the communication environment at the time of loading, the communication environment and specifications of the PC used, and the site's anti-bot measures.

Although the developer will conduct testing before delivering the site, testing on the PC that will actually be used is mandatory, so I ask for your cooperation in this testing.

The scraping programs created by Uesugi have many measures in place to ensure that they can correctly acquire information and control the browser even in unstable environments.

I can provide you with the most stable operation possible.

■If the site is bot-proofed

●Login process

Some sites have anti-bot measures in place.

Typical examples are the assembly of puzzles that appear when the login process is performed, or the selection of similar photos.

When such a process occurs, the program cannot handle it, so it must wait for the user to complete the process with the mouse.

If this operation is undesirable for the user, the program will not be able to handle it, and the development will be terminated.

In this case, no fee will be charged.

●Detection of number and frequency of connections

Some sites look at connection behaviour (number of connections, connection intervals, etc.) and place restrictions on operations and accounts.
*An easy-to-understand example would be Instagram.
　(e.g.: "Like" dozens of times in a row will disable "Like", etc.)

In case of such sites, please consult with me to change the connection interval or control interval randomly to avoid this problem.

●Access denial

Some sites deny any access by bots.

In this case, I can take some countermeasures, but there may be cases where we cannot avoid the problem, in which case we will discontinue the development.

In such cases, we will not charge any fee.

■If the configuration of the target site changes

Because scraping is used to locate and retrieve information from the HTML structure of a site and to execute controls, changes on the site side can cause the program to generate errors or fail to retrieve information correctly.

This is an unavoidable phenomenon since it is done at the will of the target site.

Uesugi makes every effort to create programs that are unaffected by changes in the site configuration, but there are limits to what can be done.

If a program becomes unavailable due to changes on the target site side, I will respond to minor modifications free of charge.
*I consider work that takes less than 15 minutes, including testing, to be a "minor modification.

If the modification requires more man-hours, please let me consult for a fee of $20 to $40(excluding tax).

Please also refer to the page below for information on how to request development.

How to request