SEO and dynamically generated pages

[Mike_FWT]

Would someone well-informed about SEO (search engine optomization) please provide specific details/advice on how to best deal with sites that are dynamically generated? For example, our site has almost no static pages and is strictly created on the fly from a database using PHP.
[list=a]
How do search engine robots traverse these pages (if at all)?
What can/should we do to improve our search rankings (right now we are very low on the SEO totem pole)?
How should we consider SEO in our future site design/updates?
[/list=a]
Thank you sincerely (in advance) for any advice...

[Adam]

Mike,

I'm no SEO expert, so I'm not going to try and act like one. But I think from a conversation I had with an SEO firm, dynamic pages (or pages utilising parameters) are less likely to be crawled, if at all by search bots. However, I frequently see dynamic pages on results when I search @ Google. But just incase, I am having our new site (which is being done by Alan Hettinger) to use the Mambo CMS which will allow us to easily manage our site while also allowing us to create what seems to be static pages.

E.g. Instead of index.php?section=about it would be about.php.

It's become more and more attractive to use CMS nowadays due to the easy usage and SEO integrated features that many already possess.

[Andrew]

Hi Mike - Im also no SEO expert - BUT - we moved our site to mambo CMS (as Adam is planning on doing) - and found the following:

- initially - NONE of our dynamic pages were being crawled (and our whole site is dynamic)
- Site titile is REALLY important
- Page titles & URL's are really important (so, if youre using mambo or something similiar make sure you install and SEF URL hack and configure to meet your parameters)
- SITE MAP (I cannot stress this enough) - Once we added a site map we went from never being crawled to what you se below- Robots .txt file - direct the bots where you want them to go - block their access to admin as well as areas of your site which will dilute your content (ie not relavent to what you want searches to find you under)

22 different robots* Hits Bandwidth Last visit
Unknown robot (identified by 'crawl') 2943+14 84.45 MB 21 Aug 2004 - 02:04
MSNBot 1682+105 43.33 MB 23 Aug 2004 - 00:27
Alexa (IA Archiver) 896+29 19.86 MB 23 Aug 2004 - 09:52
Googlebot (Google) 469+69 19.08 MB 23 Aug 2004 - 21:43
Inktomi Slurp 259+143 4.64 MB 23 Aug 2004 - 00:24
Templeton 279 1.02 MB 19 Aug 2004 - 19:22
Jeeves 105+11 13.11 MB 23 Aug 2004 - 17:13
larbin 80+16 2.03 MB 23 Aug 2004 - 06:04
ht://Dig 24+50 520.32 KB 22 Aug 2004 - 05:32
Unknown robot (identified by hit on 'robots.txt') 0+42 31.34 KB 22 Aug 2004 - 13:32

We basically went from 0 crawling to getting throughly indexed daily by making those few simple changes...We now show up in the top 1-3 pages for every 50 search terms (still working on the rest) - reality is though - dynamically generated pages will not do as well as static pages that are optimized properly - but - you can still achieve significant results none the less...

Hope this helps...

[Adam]

It's certainly helped me in terms of customising MCMS to work effectively. Thanks for the input Andrew!

[drmike]

I basically did what Andrew did but created a near hidden link in my menu to a page that listed a link to all of the articles on my site. I noticed that most of the search engines were only going three levels deep as well while the articles were on page 4. I also use a tool for my PHPNuke site that rewrites the URL into something easier for the search engines to read. Both those things helped quite a bit. It's not perfect but we're getting there.

Site and article/page name is very important as well as you just see the URL in the search engine.

-drmike

[Andrew]

Hi Adam,

Anytime ! We've been using mambo for awhile and I simply cannot say enough good things about it....

Wen you get going, feel free to contact me - theres loads of little tips Id be willing to share....

If you get the static pages portion working well - Id be interested in hearing about that as well - Ive given it a go - but, the results were messy :)

Either way - welcome to mambo and let me know how it goes...

[Adam]

Andrew,

I'll hold you to that and let you know once we're running :) Thanks for your generosity.