Everything in requests eventually goes through an adapter to do the transport work. requests-mock creates a custom adapter that allows you to predefine responses when certain URIs are called. Show
There are then a number of methods provided to get the adapter used. A simple example:
Obviously having all URLs be mock:// prefixed isn’t going to be useful, so you can use requests_mock.Mocker to get the adapter into place. One of the primary steps in attacking an internet application is enumerating hidden directories and files. Doing so can often yield valuable information that makes it easier to execute a particular attack, leaving less room for errors and wasted time. There are many tools available to try to do this, but not all of them are created equally. Gobuster, a record scanner written in Go Language, is worth searching for. In popular directories, brute-force scanners like DirBuster and DIRB work just elegantly but can often be slow and responsive to errors. Gobuster may be a Go implementation of those tools and is obtainable in a convenient command-line format. The primary benefit Gobuster has over other directory scanners is speed. As a programing language, Go is understood to be fast. It also has excellent help for concurrency, so that Gobuster can benefit from multiple threads for quicker processing. The one defeat of Gobuster, though, is the lack of recursive directory exploration. For directories, quite one level deep, another scan is going to be needed, unfortunately. Often, this is not that big of a deal, and other scanners can intensify and fill in the gaps for Gobuster in this area. Installation Steps of Gobuster Tool in Linux OSStep 1: Create a working directory to keep things neat, then change into it. ~# mkdir gobuster ~# cd gobuster/ Step 2: We need to install Gobuster Tool since it is not included on Kali Linux by default. ~/gobuster# apt-get install gobuster Step 3: Then, simply type gobuster into the terminal to run the tool for use. ~/gobuster# gobuster -h Step 4: Installing Additional Seclists for brute-forcing Directories and Files ~/gobuster# apt-get install seclists By default, Wordlists on Kali are located in the /usr/share/wordlists directory. How to use Gobuster Tool for Scanning?Gobuster tools can be launched from the terminal or command-line interface. You just have to run the command using the syntax below. gobuster [Mode][Options] Understanding Gobuster [Mode]After entering the “gobuster” command in a terminal, you compulsory need to provide the mode or need to specify the purpose of the tool you are running for. Gobuster tool have many modes :
dir mode options
dns mode options
vhost mode options Mostly, you will be using the Gobuster tool for digging directories and files. In this case, dir mode will be helpful for you. gobuster dir [options] Understanding Gobuster [Options]After entering the specific mode as per requirement, you have to specify the options. Gobuster tool has a long list of options; to explore them, you can simply read the help page by typing “gobuster -h”. You could use “gobuster dns -h” to explore options that are specifically related to the dns mode). Some examples of options are :
gobuster -h option result Target SpecificationSo, while using the tool, we need to specify the “-u” followed by a target URL, IP address, or a hostname. This option is compulsory, as there is a target specified for getting results. Some of the examples show how to use this option.
Note that these examples will not work if the mandatory option “-u” is not specified. Wordlist SpecificationGobuster Tool enumerates hidden directories and files in the target domain by performing a brute-force attack. A brute-force attack consists of matching a list of words or a combination of words hoping that the correct term is present in the list. So, Gobuster performs a brute attack. To force an attack, we need to specify a collection of words, i.e., wordlist. So to provide this wordlist, you need to type the “-w” option, followed by the path of the wordlist where it is located. We can use a wordlist file that is already present in the system.
Enumerating FilesGobuster Tool can enumerate hidden files along with the remote directories. Gobuster allows us to use the “-x” option followed by the file extensions you’d like to search for. Consider the example below:
In this command, we are specifically searching for files that have php,htm or html extensions. Usage of Gobuster Tool with an Example1. Obtaining Full Path for a directory or fileOption “-e” is used for completing printing URL when extracting any hidden file or hidden directories.
Obtaining Full Path for a directory or file 2. Hide Status CodeUsing -n Option “no status” mode prints the results’ output without presenting the status code.
Hide Status Code 3. Disable BannerGobuster tool constantly adds the banner to define the brief introduction of applied options while launching a brute force attack. By using the -q option, we can disable the flag to hide extra data.
Disable Banner 4. Set Threads NumberUsing the -t option enables the number of thread parameters to be implemented while brute-forcing sub-domain names or directories.
Set Thread Number 5. Obtain Sub Domain IPsUsing the -i option allows the IP parameter, which should show the IPs of selected sub-domains.
Obtain Sub Domain IPs 6. TimeoutUsing the –timeout option allows the timeout parameter for HTTP requests, and 5 seconds is the default time limit for the HTTP request.
Timeout 7. Appending Forward SlashI am using the -f option here for appending the forward-slash while making a brute-force attack on the target URL.
Appending Forward Slash 8. Enumerating Directory with Specific Extension ListThere are many scenarios where we need to extract the directories of a specific extension over the victim server, and then we can use the -X parameter of this scan. This parameter allows the file extension name and then explores the given extension files over the victim server or computer.
Enumerating Directory with Specific Extension List 9. Follow RedirectUsing -r options allows redirecting the parameters, redirecting HTTP requests to another, and changing the Status code for a directory or file.
Follow Redirect 10. HTTP AUTHORIZATION (-u username: password)HTTP Authentication/Authentication mechanisms are all based on the use of 401-status code and WWW-Authenticate response header. The most generally used HTTP authentication mechanisms are Primary. The client sends the user name and password un-encrypted base64 encoded data. So, to avoid this kind of authentication with the help of Gobuster, we have used the command below:
HTTP Authorization 11. Force Processing Brute ForceIt ends by obtaining the sub-domain name if it meets any Wildcard DNS, which is a non-existing domain. Therefore, it uses the –wildcard option to allow parameters to continue the attack even if there is any Wildcard Domain.
Force Processing Brute Force 12. Hide Process of ExtractingUsing the -z option covers the process of obtaining sub-domains names while making brute force attacks.
Hide Process of Extraction 13. Extracting CNAME RecordsUsing the –cn option enables the CNAME Records parameter of the obtained sub-domains and their CNAME records.
Extracting CNAME Records 14. Proxy URLUsing the –p option allows proxy URL to be used for all requests; by default, it works on port 1080. As you can see, on examining the victim’s network IP in the web browser, it put up an “Access forbidden error”, which means this web page is operating backwards by some proxy.
Proxy URL Example :Now that everything is set up and installed, we’re ready to go and use Gobuster. Let’s run it against our victim with the default parameters. Target for Scanning : https://testphp.vulnweb.com Scanning for Directories and Files
From the above screenshot, we are enumerating for directories on https://testphp.vulnweb.com. The wordlist used for the scanning is located at /usr/share/wordlists/dirb/common.txt Going to the current directory which is identified while scanning From the above screenshot, we have identified the admin panel while brute-forcing directories. After opening the web browser and typing the URL of our target, https://testphp.vulnweb.com/ and giving the identified directory /admin/, we will provide the contents available in that directory. Being a Security Researcher, you can test the functionality of that web page. ConclusionIn this article, we learned about Gobuster, a directory brute-force scanner written in the Go programming language. First, we learned how to install the tool and some valuable wordlists not found on Kali by default. Next, we ran it against our target and explored many of the varied options it ships with. Gobuster is a fast and powerful directory scanner that should be an essential part of any hacker’s collection, and now you know how to use it. |