CodeVerge.Net Beta


   Explore    Item Entry    Members      Register  Login  
NEWSGROUP
.NET
Algorithms-Data Structures
Asp.Net
C Plus Plus
CSharp
Database
HTML
Javascript
Linq
Other
Regular Expressions
VB.Net
XML

Free Download:




Zone: > NEWSGROUP > Asp.Net Forum > general_asp.net.master_pages_themes_and_navigation_controls Tags:
Item Type: NewsGroup Date Entered: 4/5/2008 3:26:03 AM Date Modified: Subscribers: 0 Subscribe Alert
Rate It:
(NR, 0)
XPoints: N/A Replies: 1 Views: 17 Favorited: 0 Favorite
Can Reply:  No Members Can Edit: No Online: Yes
2 Items, 1 Pages 1 |< << Go >> >|
zoltac007
Asp.Net User
Multiple Sitemaps for webcrawlers question4/5/2008 3:26:03 AM

0/0

I have a folder for each of the fifty states with each folder containing between 200 and 40,000 html files.   I am using the code below to generate a sitemap.aspx in each folder.  All of the filenames in each folder start with "Link".

foreach (string file in Directory.GetFiles(Server.MapPath("/State/AK/"), "Link*"))

I only need the folder sitemap to create the hyperlinks for webcrawlers.  An example is .../State/AK/Sitemap.aspx   It generates the links just fine and produces the below hyperlinks:

LinksAKANCHORAGE.html
LinksAKANCHORAGE1716.html
LinksAKANCHORAGE1734.html
LinksAKANCHORAGE1749.html
LinksAKANCHORAGE1760.html
LinksAKANCHORAGE1912.html
LinksAKANCHORAGE1913.html
LinksAKANCHORAGE2001.html
LinksAKBARROW.html

MY QUESTIONS:  1) Is creating individual sitemaps in each folder appropriate and 2) do I need to create another sitemap in the root that contains links to the individual folder specific sitemaps?  I am not using this for navigation only to help the webcrawlers find these dynamically generated html pages.

Thank you

Johnson2007
Asp.Net User
Re: Multiple Sitemaps for webcrawlers question4/7/2008 8:31:44 AM

0/0

 According to your description, do you just want to create a page that can navigate to the thousands of HTML files? If so, I suggest just one aspx in the root file is OK. Just include the sub folder information in the link. I only give a simple sample. If I have any misunderstand, please let me know.

 
 

        string strRootPath = Server.MapPath("Your root folder");
string[] strSubPaths = Directory.GetDirectories(strRootPath);

foreach (string strSubFolder in strSubPaths)
{
string[] files = Directory.GetFiles(strSubFolder, "Link*");
foreach (string file in files)
{
string strLink = strSubFolder + "/" + file;
//TODO, Generate your link.
}
}
  
Johnson
2 Items, 1 Pages 1 |< << Go >> >|


Free Download:


Web:
Help Web crawlers efficiently crawl your portal sites and Web sites May 8, 2007 ... You can also use the sitemap index file to combine multiple .... sitemaps, this will greatly improve the ability for Web crawlers to ...
Software Download: Web Crawler Sitemap Writer Pro includes built in site crawler for adding URLs into the XML ... Merge email addresses from multiple source files, removing duplicates. ...
Coding Horror: The Importance of Sitemaps Oct 12, 2008 ... But you can have multiple sitemaps in a sitemap index file, too. ..... I guess the need for sitemaps is simple: if a webcrawler starts ...
Sitemaps and Google Verify - Moonfruit Lounge These are folders that the the web crawlers (or robots) should not be accessing as they ..... Are we able to submit our sitemap to multiple search engines? ...
Google Sitemaps | Search Engine Site map | About Us Sitemaps supplement web crawlers data to allow crawlers that support Sitemaps to ... They can create one sitemap and submit it to multiple search engines ...
How we funneled searchers from Google to our collections by ... Google's protocol allows multiple sitemaps to be linked together in an index. ... we had some pretty specific questions that would be hard to answer with an ...
Sitemaps Make Your Site Search Engine Friendly | Search Engine ... Sep 30, 2008 ... This may be an obvious question but would you be submitting a ... Though you do not have to use multiple sitemaps, you may want to for ...
XML.com: Google Sitemaps It makes sense to tell a web crawler that it might want to come back for an ... for URLs to add to the Sitemap, to consolidate multiple existing Sitemap XML ...
Sitemaps - Blog Toplist Submit your Sitemaps for multiple hosts on a single dedicated host 2007-11-03 20 :39:00 Sitemaps question answered by the one who knows the best the Official ...
SEO Notes - Web Crawlers Web Crawler - Parallelization Policy. Posted by seonotes in April 19th 2006. A Crawler that runs multiple processes in parallel is a parallel crawler. ...




Search This Site:










how to - user controls

securityexception thrown by winform user control trying to access webservice

tostring() – formatting for the web

dal builder 3 and deploying a module

upload anything at filemanager >> dnn2

arrays?

new classifieds starter kit setup

dnnupdate news

fastest decimal data type?

how to serialize instance of a class to xml and than to string

what are references ?

email activation

memberlist

cannot redirect to a specfic page

the best way to store user / password for current user

browser "back" button with iframe

skinners, can you help?

throwing exception

allowing 2 users to access system w/same id and pw at the same time?

http://www.freewebdot.com/

problem when web app published but ok on asp.net development server

accessing a user's profile without logging them in

find out if a user is still connected

how do i use web.config if directory is no configured to be application?

refreshing a page from code-behind

how granular can forms authentication get ??

the mystery of the dnn 4.0.1 solution file - an assumption

lost ajax intellisense and web.config registration ability on vista sp1

client requirements for web-based applications

good private label or agent partner reseller sites

 
All Times Are GMT