Jump to content

New Sitemap module - bad data format


Recommended Posts

in file: gsitemap.php  (located in module directory) you've got code like this one:

private function _createIndexSitemap()
    {
        $sitemaps = Db::getInstance()->ExecuteS('SELECT `link` FROM `'._DB_PREFIX_.'gsitemap_sitemap` WHERE id_shop = '.$this->context->shop->id);
        if (!$sitemaps)
            return false;

        $xml = '<?xml version="1.0" encoding="UTF-8"?><sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"></sitemapindex>';
        $xml_feed = new SimpleXMLElement($xml);

        foreach ($sitemaps as $link)
        {
            $sitemap = $xml_feed->addChild('sitemap');
            $sitemap->addChild('loc', 'http://'.Tools::getShopDomain(false, true).__PS_BASE_URI__.$link['link']);
            $sitemap->addChild('lastmod', date('c'));
        }
        file_put_contents(dirname(__FILE__).'/../../'.$this->context->shop->id.'_index_sitemap.xml', $xml_feed->asXML());
    } 

there is a line with: $sitemap->addChild('lastmod', date('c'));

 

change date('c') to date ('Y-m-d')

Link to comment
Share on other sites

+ change:

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => $product->date_upd, 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

to

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => date("d-m-Y", strtotime($product->date_upd)), 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

and all other too :/

Link to comment
Share on other sites

	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

try with this

Link to comment
Share on other sites

Wait, I found a wrong format in your code:

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => date("d-m-Y", strtotime($product->date_upd)), 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

Look at date format, I am trying to generate new sitemap

Link to comment
Share on other sites

check Polish board, solution:

private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL){
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

works.

 

so conclusion:

use this: http://www.prestashop.com/forums/topic/283362-new-sitemap-module-bad-data-format/?view=findpost&p=1426113

and this: http://www.prestashop.com/forums/topic/283362-new-sitemap-module-bad-data-format/?view=findpost&p=1426185

 

nothing more ;)

  • Like 2
Link to comment
Share on other sites

OK. I think my problem is the same as described in russian forum. Sitemap doesn't work when importing products via CSV file.

 

see here :

 

http://www.prestashop.com/forums/topic/283332-prestashop-156-%D0%B8-%D0%BD%D0%BE%D0%B2%D1%8B%D0%B9-%D0%BC%D0%BE%D0%B4%D1%83%D0%BB%D1%8C-google-sitemap/?view=findpost&p=1425984

 

Can someone help me and translate in clear form what to do in english or french ?

 

Thank you !

Link to comment
Share on other sites

Hello,

 

Just did today's upgrade (V 2.3.1) and the error still exists... Google does not like the date format. I checked with two validators and the document is valid.

 

OLD date format which worked:

2013-08-25T10:13:11+02:00

 

NEW date format which doesn't work: 

2013-10-20 08:28:39

 

So who's right then?

Edited by keskifo (see edit history)
Link to comment
Share on other sites

Tried the following way with date('c') and it worked:

	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date('c', strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}
		foreach ($sitemaps as $link)
		{
			$sitemap = $xml_feed->addChild('sitemap');
			$sitemap->addChild('loc', 'http://'.Tools::getShopDomain(false, true).__PS_BASE_URI__.$link['link']);
			$sitemap->addChild('lastmod', date('c'));
		}
		file_put_contents(dirname(__FILE__).'/../../'.$this->context->shop->id.'_index_sitemap.xml', $xml_feed->asXML());
	}

gsitemap.php

Link to comment
Share on other sites

Can someone look at this and tell if it will help. My date looks fine already as it is Y-m-d. So i guess the problem is with date of last modification.

 

 

The solution by Kharkov_max from Russian forum is:

Replace

if (! $ this-> _addLinkTo Sitemap ($ link_ sitemap, array ('type' => 'product', 'page' => 'product', 'lastmod' => $ product-> date_upd, 'link' => $ url, 'image' => $ image_product), $ lang ['iso_code'], $ index, $ i, $ product_id ['id_product']))

with this

if (! $ this-> _addLinkTo Sitemap ($ link_ sitemap, array ('type' => 'product', 'page' => 'product', 'lastmod' => date ('c', strtotime ($ product- > date_upd)), 'link' => $ url, 'image' => $ image_product), $ lang ['iso_code'], $ index, $ i, $ product_id ['id_product']))

After this change, the module will generate a sitemap. Xml correct date.

These changes for the categories of suppliers and manufacturers.

 

But!

If you imported new products through CSV import and did not edited, the field from which to take the data to the module is NULL, ie date conversion to XML will give nonsense and google will swear to it.

 

What would you need to treat it in a database MySQL:

1. Current Items from the date of renovation NULL assign a date product was added 

update prefix-table_product_shop set date_upd = date_add where date_upd = '0000-00-00 00:00:00 '

2. Add a table prefix-table_product_shop and field date_upd trigger is hang on insert (appendix)

DROP TRIGGER IF EXISTS `database`. `Update_date_product_insert`;
DELIMITER $ $
CREATE TRIGGER `database`. `Update_date_product_insert`
BEFORE insert ON `database`. `Prefix-table_product_shop`
FOR EACH ROW
BEGIN
IF @ DISABLE_TRIGGERS IS NULL THEN
set NEW.date_upd = NEW.date_add;
END IF;
END $ $
DELIMITER;

Thus all the items have a date for the module triggered reporting date added when importing CSV, and edited by manual editing of the goods.

 

PS.

All of this has been tested and works, but before going live better test it on the test site

  • Like 1
Link to comment
Share on other sites

GOOD news

 

in fact it works ! Yesterday I wrote there was the same error message after uoploading the new sitemap to google. It was showing the same errors.

 

This morning, I checked at google and... errors desappear without doing or uploading anything.

 

however I clicked several time on "refresh page" on google, I think the problem comes from google cache.

 

so for those who still have problem :

 

1. delete your sitemap files in your root directory

2. download the attached file

3. replace the gsitemap.php file with the one you just downloaded

4. go in your BO / module / gsitemap and click on refresh. if necesserry click install several time

5. create your sitemap

6. go to google webmaster tools

7. delete your actual sitemap

8. click on refresh page

9. upload your new sitemap to google

7. check if errors still showing

8. wait a while (for me 24 hours)

9. go back to webmaster tolls

10. check again.

 

now the errors must ae desappeared.

gsitemap.php

  • Like 2
Link to comment
Share on other sites

Has this plugin been resolved?   Is there one post which has the all the code required for the fix, instead of piecing together multiple threads?

 

Whats the chance this module will just be fixed so we don't have to waste our time with this type of issue?

 

Seems everywhere I turn with this product (Prestashop) there is some sort of stupid issue happening....

 

-A rant is not a rant if it's true

Link to comment
Share on other sites

GOOD news

 

in fact it works ! Yesterday I wrote there was the same error message after uoploading the new sitemap to google. It was showing the same errors.

 

This morning, I checked at google and... errors desappear without doing or uploading anything.

 

however I clicked several time on "refresh page" on google, I think the problem comes from google cache.

 

so for those who still have problem :

 

1. delete your sitemap files in your root directory

2. download the attached file

3. replace the gsitemap.php file with the one you just downloaded

4. go in your BO / module / gsitemap and click on refresh. if necesserry click install several time

5. create your sitemap

6. go to google webmaster tools

7. delete your actual sitemap

8. click on refresh page

9. upload your new sitemap to google

7. check if errors still showing

8. wait a while (for me 24 hours)

9. go back to webmaster tolls

10. check again.

 

now the errors must ae desappeared.

 

 

 

Bonjour FrenchMaster,

 

Avant tout merci pour votre post Ultra Clair ... et ça marche !

 

Pour info (au cas ou pour vous et les autres utilisateurs)

 

je rencontre des problèmes lors de la création Manuelle des différents sitemaps (Multiboutique) les liens comportants des erreurs.

J'ai contourné le problème un créant une tache CRON par domaine.

 

Je suis en PS 1.5.6

 

-------------

 

J'ai essayé de désinstaller le module V 2.3.1 pour revenir à l'avant dernière version mais visiblement il n'est pas possible ( avez-vous essayé? ) et petite dernière question, avec le version 2.2.11 du module il y a fait 3 fichiers (en tout) par domaine, avec la V V 2.3.1 je n'ai plus que 2 .. c'est aussi votre cas ?

 

Merci à vous

Matias

Link to comment
Share on other sites

	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

try with this

 

I am sorry, but I don't get where should I place\replace this line ?

Link to comment
Share on other sites

I GET MAAAAAAD !

 

it doesn't work anymore for me : No problem with te date, and all my trials were OK with 100 products, but now I have 5000 products and I get a 404 page :-(

 

I had no problems with the 5000 products with the 1.5.5.0 version... 

 

I increase max_execution time but it doesn't help. 

 

Any idea ?

Link to comment
Share on other sites

It must be some sort of google cache issues.

I tried this morning to generate a new xml to replace last week's one. Surprisingly I got the date error issue and gave up.

I have restored the last week's xml but google keeps showing the same errors when testing the sitemap although it's been 3 hours now since I have replaced it with the old and good one.

Link to comment
Share on other sites

Do you think google still gonna index those pages?

Yes, but you need to change file name before you upload your file again to google. I received confirmation and 0 errors after uploading separate file with different file name. When you upload the file with same name as you uploaded previous file you will get same errors.

Link to comment
Share on other sites

from https://support.google.com/webmasters/answer/35738

----------------------------

Invalid date Your Sitemap contains one or more invalid dates.

This could be because a date is in the incorrect format, or the date itself is not valid. Dates must use W3C Datetime encoding, although you can omit the time portion if you wish. For example:

2005-02-21
2005-02-21T18:00:15+00:00

Specifying time is optional (the time defaults to 00:00:00Z), but if you do specify a time, you must also specify a timezone.

Make sure your dates match one of the following W3C Datetime formats. Then upload and resubmit your updated Sitemap.

 

--------------------------

The actual version of gsitemap 2.3.1 has a wrong date record because is without the Time zone indication.
However the old gsitemap.php (version 2.3) works very fine because the date is without the optional time zone specification.
 
Thanks to 2grosiek7 who posted 22 october 2013 - #26 the download link of the old php.gif  gsitemap.php   33.25KB to replace the newest one.
 
I think that the gsitemap version 2.3.1 have to be corrected
 
Roberto Zambelli
Link to comment
Share on other sites

Yes, but you need to change file name before you upload your file again to google. I received confirmation and 0 errors after uploading separate file with different file name. When you upload the file with same name as you uploaded previous file you will get same errors.

the default name when sitemap is generated is 1_index_sitemap.xml.  Can i change it to any name? Will google let me submit it? I just don't want to mess something up

Link to comment
Share on other sites

What if i upload master sitemap  /1_index_sitemap.xml to webmaster tools instead of /1_index_sitemap.xml ? I just did it and i got more errors as before but the date looks like this :

 

3996 errors

 

Parent tag: url

Tag: lastmod
Value: 2013-09-14 22:42:11
 
When i upload  /1_index_sitemap.xml i get date error like this:
 
1119 errors
 
Parent tag: url
Tag: lastmod
Value: -001-11-30T00:00:00-05:00
 
Not sure what to do, what the difference between those 2 urls?
 
 
 
 
Edit: Uploaded same sitemap to bing and its fine, no errors.
Edited by SeVi (see edit history)
Link to comment
Share on other sites

Im in US and here is my solution:

 

Format for date should be: "Y-m-d\TH:i:sP"

result: 2013-10-03T02:22:46-07:00

 

To have new sitemap resubmited right away I renamed file to 2_index_sitemap.xml and 2_en_0_sitemap.xml

Open file 2_index_sitemap.xml and changed link to 2_en_0_sitemap.xml

Submit new file to google. 2 minutes later files proccessed without errors.

Edited by asssets (see edit history)
Link to comment
Share on other sites

Im in US and here is my solution:

 

Format for date should be: "Y-m-d\TH:i:sP"

result: 2013-10-03T02:22:46-07:00

 

To have new sitemap resubmited right away I renamed file to 2_index_sitemap.xml and 2_en_0_sitemap.xml

Open file 2_index_sitemap.xml and changed link to 2_en_0_sitemap.xml

Submit new file to google. 2 minutes later files proccessed without errors.

 

how you changed dateformat to format that you said?

Link to comment
Share on other sites

@assetsL I don't see any Y-m-d formatting in gsitemap (my version 2.2.11).

The date is formatted by date© instruction. That should generate already 2004-02-12T15:19:21+00:00 as stated in the php manual.

Still, for some reason it doesn't.

I think that is where the issue resides.

Link to comment
Share on other sites

I used code you provided on first page, but instead of Y-m-d i used Y-m-d\TH:i:sP 

Used Ctrl+F to search y-m-d and were replacing with Y-m-d\TH:i:sP

I had 3 lines with date.

Google sitemap Version :2.3.1

attachicon.gifgsitemap.php

I just tried this:

-downloaded and replaced gsitemap.php(the date in the file already looked like this Y-m-d\TH:i:sP. Do i still need to change date somewhere?)

-deleted old sitemap and generated new one

-renamed both sitemaps

-submited to wmt

 

I have 4001 pages submited and i still get 1119 errors

Parent tag: url
Tag: lastmod
Value: -0001-11-30T00:00:00-05:00
 
Not sure what else i can do. Gonna wait for 24 hours and see if errors will disapear but i doubt they will.
Edited by SeVi (see edit history)
Link to comment
Share on other sites

i have no idea. Just checked and some of the urls have date  2013-09-14T22:42:11-04:00 and some -0001-11-30T00:00:00-05:00.

 

Doesn't make any sense why it would generate only some of it with correct date format.

Did you check data in database?

Link to comment
Share on other sites

i checked same product that has bad date in sitemap. In database it has only  date_add and date_upd and it looks fine

2013-09-28 03:40:31 2013-09-30 00:45:59  

 

Where do i find last mod of the product in DB?

 

this field name is located in ps_product_Shop table and field name is: date_upd

Link to comment
Share on other sites

So, I also had this problem and read all of the comments in this topic.

 

Give it a try. WT now accepts the file with no errors. But if you check the date format in the file before sending it in WT it won't be the date you've created the file. I suppose that with this file now it just ignores the date:

1. Download the attached file - gsitemap.php.

2. Delete previous sitemaps on your server.

3. Deactivate the module in BO.

4. Activate the module again.

5. Generate the sitemap(s).

6. Rename the generated file (2 files if you have multiple languages).

7. Upload the file in WT

8. Check for errors now.

 

Any success?

  • Like 1
Link to comment
Share on other sites

i just searched my DB for products that have bad date format in the sitemap. The ones with bad date have different values from "normal" products. 

fields:

redirect_type is empty(other products have 404)

indexed = 0 

date_add looks fine, but date_upd is 0000-00-00 00:00:00

 

Weird thing. When i open the product with empty fields in BO and click on save, without editing anything it changes values in the database. So what will be the quickest way to fix this? It will take long time to open and save them one by one.

 

Update:

 

I have added not indexed products to index in Preferences>Search>Add missing products to the index.

it fixed the indexed field, but not redirect_type and date_upd

 

 

Update 2

I dont get it, just ran sql query to replace all 00-00-00 date_upd values to todays date. It replaced everything just fine but in sitemap those products are shown with <lastmod>1969-12-31</lastmod>

Edited by SeVi (see edit history)
Link to comment
Share on other sites

  • 3 weeks later...
  • 2 weeks later...

So, I also had this problem and read all of the comments in this topic.

 

Give it a try. WT now accepts the file with no errors. But if you check the date format in the file before sending it in WT it won't be the date you've created the file. I suppose that with this file now it just ignores the date:

1. Download the attached file - attachicon.gifgsitemap.php.

2. Delete previous sitemaps on your server.

3. Deactivate the module in BO.

4. Activate the module again.

5. Generate the sitemap(s).

6. Rename the generated file (2 files if you have multiple languages).

7. Upload the file in WT

8. Check for errors now.

 

Any success?

 

After application of your suggestions, it now works. Thank you!

  • Like 1
Link to comment
Share on other sites

  • 2 months later...

Picking into this...

 

Anyone having a Google Sitemap version for download compatible with PS version 1.4.9? By default I had gsitemap v1.9 in my shop but it also suffered the dateformat error. Version 2.3.2 isn't compatible with PS 1.4.9.  

Link to comment
Share on other sites

  • 2 weeks later...

Picking into this...

 

Anyone having a Google Sitemap version for download compatible with PS version 1.4.9? By default I had gsitemap v1.9 in my shop but it also suffered the dateformat error. Version 2.3.2 isn't compatible with PS 1.4.9.  

 

You can download older presta versions here http://www.prestashop.com/en/developers-versions and just use module from there

Link to comment
Share on other sites

  • 1 month later...
  • 1 month later...
  • 4 weeks later...
  • 1 year later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...