|
robots.txt - sprijeciti indexiranje odredjnih linkova
Zdravo.
Ne znam gdje da postavim temu, a i pitanje je dosta početničko :) U robots.txt stavio sam: Kôd:
User-agent: *Gdje grijesim? Hvala i pozdrav. |
mislim da pretrazivaci ne prepoznaju * u Disallow.
ukratko - sa robots.txt neces uspeti to da uradis. stavi nofollow u linkovima ka tim stranicama, a preko google webmaster tools obrisi te indeksirane stranice. |
Stavio sam rel="nofollow" u te linkove, ali čini mi se da je google ipak prepoznao te linkove kao zabrajene jer u "URL restricted by robots.txt" u google webmasters tool pise da je zabranjeno nekih 120 linkova, a indexirano je svega nekih 10-tak takvih linkova. Ti linkovi koji su indexirani su takodjer u listi Restricted by robots.txt.
E sad se ne mogu sjetiti 100%, ali mislim da sam naknado dodao ova pravila u robots.txt pa da je google uspio da indexira 10-tak tih linkova u jednom danu. Hvala na pomoci. Iskren da budem nisam nikada do sada korostio google webmasters tool. Pozdrav. |
|
Pattern matching
Yes, Googlebot interprets some pattern matching. This is an extension of the standard, so not all bots may follow it. Matching a sequence of characters using * You can use an asterisk (*) to match a sequence of characters. For instance, to block access to all subdirectories that begin with private, you could use the following entry: User-agent: Googlebot Disallow: /private*/ To block access to all URLs that include a question mark (?), you could use the following entry: User-agent: * Disallow: /*? To block access to all URLs containing the word "private", you could use: User-agent: * Disallow: /*private* Matching the end characters of the URL using $ You can use the $ character to specify matching the end of the URL. For instance, to block an URLs that end with .asp, you could use the following entry: User-agent: Googlebot Disallow: /*.asp$ You can use this pattern matching in combination with the Allow directive. For instance, if a ? indicates a session ID, you may want to exclude all URLs that contain them to ensure Googlebot doesn't crawl duplicate pages. But URLs that end with a ? may be the version of the page that you do want included. For this situation, you can set your robots.txt file as follows: User-agent: * Allow: /*?$ Disallow: /*? The Disallow:/ *? line will block any URL that includes a ? (more specifically, it will block any URL that begins with your domain name, followed by any string, followed by a question mark, followed by any string). The Allow: /*?$ line will allow any URL that ends in a ? (more specifically, it will allow any URL that begins with your domain name, followed by a string, followed by a ?, with no characters after the ?). Izvor: http://www.google.com/support/webmas...n&answer=40367 |
ovo meni treba :)
tnx. |
Zna li neko zasto Google indexira stranice u kojima je u okviru head tagova stavljeno <meta name="robots" content="noindex" /> ?
Format tih adresa je: http://domen.com/forum/viewtopic.php...t=0&view=print I u robots.txt sam stavio Disallow: /forum/*&start=0&view=print i Disallow: /forum/*view=print ali dzaba, jer u Google Webmasters Centar pod 'Restricted by robots.txt' ih nema. Imali li kakva fora za masnovo slanje zahtijevaza birsanje u Google Webmasters Centru, tipa da obrise sve linkove koje u sebi sadrze 'print' ili se mora jedan po jedan :D? EDIT: I kada posaljem zatijev da se ukloni iz pretrage jedan od tih linkova, oni budu ukoljeni, a inace da bi bili uklonjeni moraju biit ili 404 ili noindex ili restriced by robots.txt |
| Vreme je GMT +2. Trenutno vreme je 17:09. |
Powered by vBulletin® Verzija 3.6.8
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © DevProTalk. All Rights Reserved.