Fomete ea robots.txt e bolokiloe motso oa websaeteng ea hau e tla bolela liroboto tsa marang-rang joaloka li-spiders tsa ho batla lipatlisiso le lifaele tseo li lumelloa ho li khasa. Ho bonolo ho sebelisa faele ea robots.txt, empa ho na le lintho tseo u lokelang ho li hopola:
- Liroboto tsa li-black hat li tla hlokomoloha faele ea robots.txt ea hau. Mefuta e tloaelehileng ka ho fetisisa ke bots le li-robot tsa malware tse batlang liaterese tsa imeile ho kotula.
- Bangoli ba bang ba bacha ba tla ngola liroboto tse hlokomolohang fono ea robots.txt. Hangata sena se etsoa ka phoso.
- Motho leha e le ofe a ka bona faele ea robots.txt ea hau. Ka linako tsohle li bitsoa liroboto.txt 'me li lula li bolokiloe motsoong oa marang-rang.
- Qetellong, haeba motho a hokahanya le faele kapa mohala o sa kenyelletsoeng ke faele ea hau ea robots.txt ho tloha leqepheng le sa kenyelletsoeng ke faele ea robots.txt ea bona, liwebsaete li ka li fumana.
U se ke ua sebelisa li-file tsa robots.txt ho pata ntho leha e le efe ea bohlokoa. Ho e-na le hoo, o lokela ho beha tlhahisoleseding ea bohlokoa ka morao ho marwedi a sireletsehileng kapa ho e tlohela ho web ka ho feletseng.
Mokhoa oa ho Sebelisa Li-Files tsena tsa Mehlala
Kopitsa temana eo ho tloha sampong e haufi le seo u batlang ho se etsa, ebe ue kenya ka faele ea hau ea robots.txt. Fetola robot, mohala, le mabitso a lifaele ho bapisa mokhoa o khethiloeng.
Li-Robot tse peli tsa motheo.Txt Files
Moemeli oa moetsi: *
Qoba: /
Feshene ena e re robot leha e le efe (User-agent: *) e fihlelang eona e lokela ho hlokomoloha leqephe le leng le le leng setšeng (Disallow: /).
Moemeli oa moetsi: *
Qoba:
Feshene ena e re robot leha e le efe (Moetsi-moetsi: *) ea fihlelang ho lumelloa ho sheba leqephe le leng le le leng setšeng (Disallow:).
U ka boela ua etsa sena ka ho tlohela faele ea hau ea robot.txt e se na letho kapa u se na sethaleng sa hau ho hang.
Sireletsa Libuka tse Hlahang ho tloha Liroboto
Moemeli oa moetsi: *
Hanela: / cgi-bin /
Qoba: / temp /
Feshene ena e re robot leha e le efe (User-agent: *) e fihlelang eona e lokela ho hlokomoloha litaelo / cgi-bin / and / temp / (Disallow: / cgi-bin / Disallow: / temp /).
Sireletsa Maqephe a Khethehileng a Tsoang Liroboto
Moemeli oa moetsi: *
Hanela: /jenns-stuff.htm
Qoba: /private.php
Feshene ena e re robot leha e le efe (User-agent: *) e kenang e lokela ho iphapanyetsa lifaele /jenns-stuff.htm le /private.php (Disallow: /jenns-stuff.htm Qoba: /private.php).
Thibela Robot e tobileng ho Fihlella Sebaka sa Hao
Moemeli-moemeli: Lycos / xx
Qoba: /
Faele ena e re bot bot Lycos (User-agent: Lycos / xx) ha e lumelloe ho fihlella kae kapa efe setšeng (Disallow: /).
Lumella ho fumana Robot e le 'Ngoe feela
Moemeli oa moetsi: *
Qoba: /
Moemeli oa basebelisi: Googlebot
Qoba:
Fifa ena pele e tlosa liroboto tsohle joaloka rona ka holimo, ebe joale re lumella Googlebot (Moemeli-Moetsi: Googlebot) ho fumana ntho e 'ngoe le e' ngoe (Disallow:).
Kopanya Linaele Tse 'maloa ho Fumana hantle-ntle seo u se batlang
Ha ho ntse ho le molemo ho sebelisa molaetsa oa basebelisi ba molekane-kakaretso, joaloka motlatsi oa basebelisi: *, o ka ba o tobileng kamoo o ratang kateng. Hopola hore liroboto li bala file ka tatellano. Kahoo haeba mela e qalang e bolela hore liroboto tsohle li thibehile ho tsohle, ebe hamorao li fihlella lefapheng le re liroboto tsohle li lumelloa ho fihlella ntho e 'ngoe le e' ngoe, liroboto li tla ba le phihlelo ea ntho e 'ngoe le e' ngoe.
Haeba o sa tsebe hantle hore na u ngotse file ea robots.txt ka nepo, u ka sebelisa lisebelisoa tsa Webmaster tsa Google ho hlahloba faele ea robots.txt kapa ho ngola e ncha.