Web Fundamentals

How do we load websites?

When you go to any website, a DNS request is made. DNS or the domain name system is the phonebook of internet and it translates URL to IP addresses so browsers can load internet resources.

An IP address is an unique address that identifies each internet connected device. It is formed of 4 groups of numbers ranged between 0-255 and each group of number is called an octet. Example: 127.0.0.1

Once you go to a browser and enter URL or IP address of the server, it can ask the server for the web page and it's done with a HTTP GET request. Then, the server will respond to a HTTP GET request by showing the web page content. Extra resources like JavaScript, images, or CCS files will be retrieved in separate GET requests.

Most websites are now often using HTTPS, more secure encrypted version of HTTP. By default, HTTP runs on port 8 and HTTPS on port 443.

A web server is a software that retrieves or responds to HTTP(S) requests. Apache, Nginx and Microsoft's IIS are the examples of it.

A web page is normally formed of HTML, CSS and JavaScript.

HTML: the structure of the page, and the content

CSS: change how the page looks and make it look fancy

JavaScript: a programming language that runs in the browser and makes pages interactive or loads extra content.

HTTP-verbs and request formats

There are 9 different HTTP verbs, each of which functions differently. We're going to learn only some because rest of it aren't as commonly used for web servers.

We already learned a GET verb, which allows you to retrieve content.

POST requests are used to send data to a web server. For example, when you log into a website or add comment on a website.

A HTTP request can be broken into 3 main parts.

GET /main.js HTTP/1.1

Host: 192.168.170.129:8081

Connection: keep-alive

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36

Accept: */*

Referer: http://192.168.170.129:8081/

Accept-Encoding: gzip, deflate

Accept-Language: en-GB,en-US;q=0.9,en;q=0.8

As you can see the GET request, the first line is a HTTP verb and a path for the server. The next section is called a header which gives you more information about your request such as type of browsers, window versions, cookies, etc. Finally, a body of the request. For POST, this is the content that's sent to the server while, for GET, it's most likely ignored by the server.

Once you send a HTTP(S) request to the server, the server will reply with a response. The structure of a response is pretty similar to that of a request.

HTTP/1.1 200 OK

Accept-Ranges: bytes

Content-Length: 28

Content-Type: application/javascript; charset=utf-8

Last-Modified: Wed, 12 Feb 2020 12:51:44 GMT

Date: Thu, 27 Feb 2020 21:47:30 GMT

console.log("Hello, World!")

As you can see, the response gives you a status code in the first line.

Here's the basic breakdown of status codes:

• 100-199: Information

• 200-299: Successes (200 OK is the "normal" response for a GET)

• 300-399: Redirects (the information you want is elsewhere)

• 400-499: Client errors (You did something wrong, like asking for something that doesn't exist)

• 500-599: Server errors (The server tried, but something went wrong on their side)

To get to know more about this, check this website: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status

Response headers can be very important. They can often tell you something about the web server sending them, or give you cookies that may prove useful later on.

The response will also have a body. For GET requests, this is normally web content or information such as JSON. For POST requests, it may be a status message or similar.

Cookies

Cookies are text files with small pieces of data that are stored in your browser. Each browser stores cookies separately. The main uses of this are either session management or advertising (tracking cookies).

The reason why cookies are used is because the HTTP is stateless, meaning that each request is independent and no state is tracked internally. Therefore, cookies allow sites to keep track of data on your browser.

Cookies have a name, a value, an expiry date, and a path. The name identifies the cookie, the value is where data is stored, an expiry date is when the browser will get rid of the cookie automatically, and a path determines what requests the cookie will be sent with.

You can manipulate cookies using your browser's developer tools.

cURL

cURL is a command line tool that uses URL syntax to transfer data to or from servers.

By default, cURL will perform GET requests on whatever URL you supply it.

cURL does not store cookies, so you have to manually specify cookies and values that you would like to send with your requests.

-How to make a GET request with cURL: curl <path>/ctf/get

-How to make a GET request with cURL: curl -X POST --data "flag_please" <path>/ctf/post

*The -X flag allows us to specify the request type, -eg -X POST. Also, specify the data to POST with --data to make it default to plain text data.

-How to make a GET request with cURL: curl -c - <path>/ctf/getcookie

*-c allows us to specify to which file you want curl to write all cookies after a complete operation.

*or, you can check the server by typing <path>/ctf/getcookie and F12 -> go to storage -> check the cookies.

How to make a GET request with cURL: curl -v --cookie <path>/ctf/sendcookie

*curl -b 'name=value' <path>/ctf/sendcookie

*or, go to <path>/ctf/sendcookie -> devtools -> go to storage -> go to cookies -> change both name and value to 'flagpls' -> F5 to refresh