Skip to main content

Find Secret Websites And Articles



There are a lot of sites on the Internet whose site owners that like to hide some of the site's ages or even the entire site from the search engines. You can now find those sites with robots.txt.


Robots.txt
is a text file present in the root directory of a site which is used to control which pages are indexed by a robot. If you use the 'disallow' word you can block parts of your sites to be found by search engines.

Step 1. Open the link http://www.google.com/ and search for the keyword :

"robots.txt" "disallow:" filetype:txt


Step 2. You will find the robots.txt file from sites that uses disallow command in it.

Step 3. Let's open for example the first site: WhiteHouse. We can see that a lot of pages were made invisible.

Step 4. To open 'forbidden' pages just copy the text from what disallow command you want, without the "text" at the end.

Step 5. Now replace in the browser /robots.txt with your copied text and press Enter. The page will open.


This is the hidden page from WhiteHouse.

There are many more interesting sites which you can unlock using this tutorial. Happy unlocking!

Comments

Popular posts from this blog

Cheap & Easy To Use VPN Services in China for 2019

Because of copyright issues, many domestic video sites prohibit foreign direct access to foreign IPs. Overseas users want to see these videos from abroad and need to obtain domestic IP with VPNs to bypass these copyright restrictions. Wall mom brings the best solutions for your VPN Services which are not only the best in the area but also have cheap and affordable prices. In this article,  Wall mom  brings some cheap and easy to use over the wall VPN recommendations for their users with the reviews which will surely help those who want a access to the sites which has the copyright restrictions.  They wrote in this articles that Only recommend foreign VPN, do not recommend domestic VPN Dare to operate in the domestic VPN service, the policy risk is particularly large, almost all shut down or run, it is difficult to issue a refund. Before GreenVPN was shut down, a large number of users flocked for a refund, and the refund was still good. Many domestic VPNs...

What are the HTML Tags?

The simplest HTML tags arrange text into blocks, designate the font or letter-style, and size of the type. Bold, italic and underline are each identified within an HTML tag that precedes the letter or word(s) to be effected. The tag appears again at the end of the chosen text with a slash to indicate the effect ends there. For example, see how the following sentence appears in HTML: HTML tags are the core of Hypertext Markup Language. HTML tags are the core of Hypertext Markup Language. The “b” stands for bold and HTML tags are always enclosed in angled brackets, with the closing tag starting with a forward slash. If the forward slash is forgotten, the remainder of the page will assume the effect. In addition to changing fonts, HTML tags also create hyperlinks, or clickable text. The hyperlink tag includes an embedded website address (URL or Uniform Resource Locator). By placing a hyperlink tag around a phrase or name, clicking on it will take the surfer to the desired address. This c...

ملگجی شام پھر نہیں آئی

ملگجی شام پھر نہیں آئی لے کے دامن میں اپنے رسوائی کھو چکے ہیں متاع جان و جگر نزہت بزم کے تمنائی ہے ہر اک روپ میں ترا پرتو تُو تو نکلا بڑا ہی ہرجائی تیرگی وہ ملی شب فرقت جس سے نرگس بھی خوب شرمائی یوں کھلے ہیں دریچے یادوں کے بڑھ گیا اور شوق تنہائی چھوڑ کر آج لالہ و گل کو بوئے گلشن بھی کتنی پچھتائی زیست کی پُرفریب راہوں پر دل نے چاہی ہے پھر شناسائی ہے اسیری جو شہر خاموشاں ہم نے دانش نہ جاوداں پائی