No matter how much I emphasise on the importance of python in security analysis, it is never enough. Being an old school, I like to run my tasks on black terminals and the streaming green texts give me a Matrix-like feel. Unless a GUI is flexible enough to allow me embed my own scripts into it (think of an advancement of Trinket), I find pure happiness in a command prompt.
As the much dreaded COVID-19 seems to be on a spree to occupy the world, Work From Home is our only option to build, teach, learn and grow. Last night, I was pen-testing a Web Application for a client (of-course remotely!), when I felt the need to automate some tasks. Due to a signed NDA, I am unable to upload the actual python script that automated few web-scraping tasks for me.
But I consider it prudent to share with our viewers, a skeleton of the script that can search for robots.txt using Google Search Engine. The complete code can be downloaded from my Github repository here.
People familiar with python basics must be able to make complete sense out this. For those who are new to Python may follow our page and reach out for additional guidance. Let us learn together.
