Jump to content

New Sitemap module - bad data format

2grosiek7
 Share

Recommended Posts

2grosiek7 Newbie

2grosiek7

Posted
Posted

Hi,

Today I was upgraded the sitemap module in prestashop. I made new sitemaps and sent it to Google Webmaster Tools. And Google gave me an error about bad data format. Who can give me a solution to solve this problem?

Regards!

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

in file: gsitemap.php  (located in module directory) you've got code like this one:

private function _createIndexSitemap()
    {
        $sitemaps = Db::getInstance()->ExecuteS('SELECT `link` FROM `'._DB_PREFIX_.'gsitemap_sitemap` WHERE id_shop = '.$this->context->shop->id);
        if (!$sitemaps)
            return false;

        $xml = '<?xml version="1.0" encoding="UTF-8"?><sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"></sitemapindex>';
        $xml_feed = new SimpleXMLElement($xml);

        foreach ($sitemaps as $link)
        {
            $sitemap = $xml_feed->addChild('sitemap');
            $sitemap->addChild('loc', 'http://'.Tools::getShopDomain(false, true).__PS_BASE_URI__.$link['link']);
            $sitemap->addChild('lastmod', date('c'));
        }
        file_put_contents(dirname(__FILE__).'/../../'.$this->context->shop->id.'_index_sitemap.xml', $xml_feed->asXML());
    } 

there is a line with: $sitemap->addChild('lastmod', date('c'));

 

change date('c') to date ('Y-m-d')

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

+ change:

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => $product->date_upd, 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

to

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => date("d-m-Y", strtotime($product->date_upd)), 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

and all other too :/

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted 
	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

try with this

Share this post

Link to post
Share on other sites
2grosiek7 Newbie

2grosiek7

Posted
Posted

Wait, I found a wrong format in your code:

if (!$this->_addLinkToSitemap($link_sitemap, array('type' => 'product', 'page' => 'product', 'lastmod' => date("d-m-Y", strtotime($product->date_upd)), 'link' => $url, 'image' => $image_product), $lang['iso_code'], $index, $i, $product_id['id_product']))

Look at date format, I am trying to generate new sitemap

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

check Polish board, solution:

private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL){
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

works.

 

so conclusion:

use this: http://www.prestashop.com/forums/topic/283362-new-sitemap-module-bad-data-format/?view=findpost&p=1426113

and this: http://www.prestashop.com/forums/topic/283362-new-sitemap-module-bad-data-format/?view=findpost&p=1426185

 

nothing more ;)

  • Like 2

Share this post

Link to post
Share on other sites
FrenchMaster Newbie

FrenchMaster

Posted
Posted

OK. I think my problem is the same as described in russian forum. Sitemap doesn't work when importing products via CSV file.

 

see here :

 

http://www.prestashop.com/forums/topic/283332-prestashop-156-%D0%B8-%D0%BD%D0%BE%D0%B2%D1%8B%D0%B9-%D0%BC%D0%BE%D0%B4%D1%83%D0%BB%D1%8C-google-sitemap/?view=findpost&p=1425984

 

Can someone help me and translate in clear form what to do in english or french ?

 

Thank you !

Share this post

Link to post
Share on other sites
keskifo Newbie

keskifo

Posted
Posted (edited)

Hello,

 

Just did today's upgrade (V 2.3.1) and the error still exists... Google does not like the date format. I checked with two validators and the document is valid.

 

OLD date format which worked:

2013-08-25T10:13:11+02:00

 

NEW date format which doesn't work: 

2013-10-20 08:28:39

 

So who's right then?

Edited by keskifo (see edit history)

Share this post

Link to post
Share on other sites
keskifo Newbie

keskifo

Posted
Posted

Tried the following way with date('c') and it worked:

	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date('c', strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

		foreach ($sitemaps as $link)
		{
			$sitemap = $xml_feed->addChild('sitemap');
			$sitemap->addChild('loc', 'http://'.Tools::getShopDomain(false, true).__PS_BASE_URI__.$link['link']);
			$sitemap->addChild('lastmod', date('c'));
		}
		file_put_contents(dirname(__FILE__).'/../../'.$this->context->shop->id.'_index_sitemap.xml', $xml_feed->asXML());
	}

gsitemap.php

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

really weird :( when you try to use this url in google account, you've got information about wrong file format? about <lastmod> field?


i just wondering, maybe the problem in your case is somewhere else, not in <lastmod> field

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted (edited)

have same issue as FrenchMaster, already tried all fixes posted in this thread. Still getting this error in webmaster tools:

 

Sitemap:mydomain.com/store/1_en_0_sitemap.xml
Parent tag: url
Tag: lastmod
Value: 2013-09-14 22:42:30
 
Does the solution from the Russian forum works?
Edited by SeVi (see edit history)

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted

Can someone look at this and tell if it will help. My date looks fine already as it is Y-m-d. So i guess the problem is with date of last modification.

 

 

The solution by Kharkov_max from Russian forum is:

Replace

if (! $ this-> _addLinkTo Sitemap ($ link_ sitemap, array ('type' => 'product', 'page' => 'product', 'lastmod' => $ product-> date_upd, 'link' => $ url, 'image' => $ image_product), $ lang ['iso_code'], $ index, $ i, $ product_id ['id_product']))

with this

if (! $ this-> _addLinkTo Sitemap ($ link_ sitemap, array ('type' => 'product', 'page' => 'product', 'lastmod' => date ('c', strtotime ($ product- > date_upd)), 'link' => $ url, 'image' => $ image_product), $ lang ['iso_code'], $ index, $ i, $ product_id ['id_product']))

After this change, the module will generate a sitemap. Xml correct date.

These changes for the categories of suppliers and manufacturers.

 

But!

If you imported new products through CSV import and did not edited, the field from which to take the data to the module is NULL, ie date conversion to XML will give nonsense and google will swear to it.

 

What would you need to treat it in a database MySQL:

1. Current Items from the date of renovation NULL assign a date product was added 

update prefix-table_product_shop set date_upd = date_add where date_upd = '0000-00-00 00:00:00 '

2. Add a table prefix-table_product_shop and field date_upd trigger is hang on insert (appendix)

DROP TRIGGER IF EXISTS `database`. `Update_date_product_insert`;
DELIMITER $ $
CREATE TRIGGER `database`. `Update_date_product_insert`
BEFORE insert ON `database`. `Prefix-table_product_shop`
FOR EACH ROW
BEGIN
IF @ DISABLE_TRIGGERS IS NULL THEN
set NEW.date_upd = NEW.date_add;
END IF;
END $ $
DELIMITER;

Thus all the items have a date for the module triggered reporting date added when importing CSV, and edited by manual editing of the goods.

 

PS.

All of this has been tested and works, but before going live better test it on the test site

  • Like 1

Share this post

Link to post
Share on other sites
FrenchMaster Newbie

FrenchMaster

Posted
Posted

GOOD news

 

in fact it works ! Yesterday I wrote there was the same error message after uoploading the new sitemap to google. It was showing the same errors.

 

This morning, I checked at google and... errors desappear without doing or uploading anything.

 

however I clicked several time on "refresh page" on google, I think the problem comes from google cache.

 

so for those who still have problem :

 

1. delete your sitemap files in your root directory

2. download the attached file

3. replace the gsitemap.php file with the one you just downloaded

4. go in your BO / module / gsitemap and click on refresh. if necesserry click install several time

5. create your sitemap

6. go to google webmaster tools

7. delete your actual sitemap

8. click on refresh page

9. upload your new sitemap to google

7. check if errors still showing

8. wait a while (for me 24 hours)

9. go back to webmaster tolls

10. check again.

 

now the errors must ae desappeared.

gsitemap.php

  • Like 2

Share this post

Link to post
Share on other sites
skibbi Newbie

skibbi

Posted
Posted

Has this plugin been resolved?   Is there one post which has the all the code required for the fix, instead of piecing together multiple threads?

 

Whats the chance this module will just be fixed so we don't have to waste our time with this type of issue?

 

Seems everywhere I turn with this product (Prestashop) there is some sort of stupid issue happening....

 

-A rant is not a rant if it's true

Share this post

Link to post
Share on other sites
ILMP Newbie

ILMP

Posted
Posted

GOOD news

 

in fact it works ! Yesterday I wrote there was the same error message after uoploading the new sitemap to google. It was showing the same errors.

 

This morning, I checked at google and... errors desappear without doing or uploading anything.

 

however I clicked several time on "refresh page" on google, I think the problem comes from google cache.

 

so for those who still have problem :

 

1. delete your sitemap files in your root directory

2. download the attached file

3. replace the gsitemap.php file with the one you just downloaded

4. go in your BO / module / gsitemap and click on refresh. if necesserry click install several time

5. create your sitemap

6. go to google webmaster tools

7. delete your actual sitemap

8. click on refresh page

9. upload your new sitemap to google

7. check if errors still showing

8. wait a while (for me 24 hours)

9. go back to webmaster tolls

10. check again.

 

now the errors must ae desappeared.

 

 

 

Bonjour FrenchMaster,

 

Avant tout merci pour votre post Ultra Clair ... et ça marche !

 

Pour info (au cas ou pour vous et les autres utilisateurs)

 

je rencontre des problèmes lors de la création Manuelle des différents sitemaps (Multiboutique) les liens comportants des erreurs.

J'ai contourné le problème un créant une tache CRON par domaine.

 

Je suis en PS 1.5.6

 

-------------

 

J'ai essayé de désinstaller le module V 2.3.1 pour revenir à l'avant dernière version mais visiblement il n'est pas possible ( avez-vous essayé? ) et petite dernière question, avec le version 2.2.11 du module il y a fait 3 fichiers (en tout) par domaine, avec la V V 2.3.1 je n'ai plus que 2 .. c'est aussi votre cas ?

 

Merci à vous

Matias

Share this post

Link to post
Share on other sites
PiterKa Newbie

PiterKa

Posted
Posted 
	private function _addSitemapNode($fd, $loc, $priority, $change_freq, $last_mod = NULL)
	{
		fwrite($fd, '<loc>'.(Configuration::get('PS_REWRITING_SETTINGS') ? '<![CDATA['.$loc.']]>' : $loc).'</loc>'."\r\n".'<priority>'."\r\n".number_format($priority, 1, '.', '').'</priority>'."\r\n".($last_mod ? '<lastmod>'.date("Y-m-d", strtotime($last_mod)).'</lastmod>' : '')."\r\n".'<changefreq>'.$change_freq.'</changefreq>'."\r\n");
	}

try with this

 

I am sorry, but I don't get where should I place\replace this line ?

Share this post

Link to post
Share on other sites
FrenchMaster Newbie

FrenchMaster

Posted
Posted

I GET MAAAAAAD !

 

it doesn't work anymore for me : No problem with te date, and all my trials were OK with 100 products, but now I have 5000 products and I get a 404 page :-(

 

I had no problems with the 5000 products with the 1.5.5.0 version... 

 

I increase max_execution time but it doesn't help. 

 

Any idea ?

Share this post

Link to post
Share on other sites
Rhobur Enthusiast

Rhobur

Posted
Posted

It must be some sort of google cache issues.

I tried this morning to generate a new xml to replace last week's one. Surprisingly I got the date error issue and gave up.

I have restored the last week's xml but google keeps showing the same errors when testing the sitemap although it's been 3 hours now since I have replaced it with the old and good one.

Share this post

Link to post
Share on other sites
PiterKa Newbie

PiterKa

Posted
Posted

hello

 

it's weird that for some of us it works well and for others not 

I am not sure if it helps, but I renamed the file name prior to uploading as google was reporting same errors if file name was unchanged.

Share this post

Link to post
Share on other sites
PiterKa Newbie

PiterKa

Posted
Posted

Do you think google still gonna index those pages?

Yes, but you need to change file name before you upload your file again to google. I received confirmation and 0 errors after uploading separate file with different file name. When you upload the file with same name as you uploaded previous file you will get same errors.

Share this post

Link to post
Share on other sites
Roberto Zambelli Newbie

Roberto Zambelli

Posted
Posted

from https://support.google.com/webmasters/answer/35738

----------------------------

Invalid date Your Sitemap contains one or more invalid dates.

This could be because a date is in the incorrect format, or the date itself is not valid. Dates must use W3C Datetime encoding, although you can omit the time portion if you wish. For example:

2005-02-21
2005-02-21T18:00:15+00:00

Specifying time is optional (the time defaults to 00:00:00Z), but if you do specify a time, you must also specify a timezone.

Make sure your dates match one of the following W3C Datetime formats. Then upload and resubmit your updated Sitemap.

 

--------------------------

The actual version of gsitemap 2.3.1 has a wrong date record because is without the Time zone indication.
However the old gsitemap.php (version 2.3) works very fine because the date is without the optional time zone specification.
 
Thanks to 2grosiek7 who posted 22 october 2013 - #26 the download link of the old php.gif  gsitemap.php   33.25KB to replace the newest one.
 
I think that the gsitemap version 2.3.1 have to be corrected
 
Roberto Zambelli

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted

Yes, but you need to change file name before you upload your file again to google. I received confirmation and 0 errors after uploading separate file with different file name. When you upload the file with same name as you uploaded previous file you will get same errors.

the default name when sitemap is generated is 1_index_sitemap.xml.  Can i change it to any name? Will google let me submit it? I just don't want to mess something up

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted (edited)

What if i upload master sitemap  /1_index_sitemap.xml to webmaster tools instead of /1_index_sitemap.xml ? I just did it and i got more errors as before but the date looks like this :

 

3996 errors

 

Parent tag: url

Tag: lastmod
Value: 2013-09-14 22:42:11
 
When i upload  /1_index_sitemap.xml i get date error like this:
 
1119 errors
 
Parent tag: url
Tag: lastmod
Value: -001-11-30T00:00:00-05:00
 
Not sure what to do, what the difference between those 2 urls?
 
 
 
 
Edit: Uploaded same sitemap to bing and its fine, no errors.
Edited by SeVi (see edit history)

Share this post

Link to post
Share on other sites
asssets Newbie

asssets

Posted
Posted (edited)

Im in US and here is my solution:

 

Format for date should be: "Y-m-d\TH:i:sP"

result: 2013-10-03T02:22:46-07:00

 

To have new sitemap resubmited right away I renamed file to 2_index_sitemap.xml and 2_en_0_sitemap.xml

Open file 2_index_sitemap.xml and changed link to 2_en_0_sitemap.xml

Submit new file to google. 2 minutes later files proccessed without errors.

Edited by asssets (see edit history)

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

Im in US and here is my solution:

 

Format for date should be: "Y-m-d\TH:i:sP"

result: 2013-10-03T02:22:46-07:00

 

To have new sitemap resubmited right away I renamed file to 2_index_sitemap.xml and 2_en_0_sitemap.xml

Open file 2_index_sitemap.xml and changed link to 2_en_0_sitemap.xml

Submit new file to google. 2 minutes later files proccessed without errors.

 

how you changed dateformat to format that you said?

Share this post

Link to post
Share on other sites
Rhobur Enthusiast

Rhobur

Posted
Posted

@assetsL I don't see any Y-m-d formatting in gsitemap (my version 2.2.11).

The date is formatted by date© instruction. That should generate already 2004-02-12T15:19:21+00:00 as stated in the php manual.

Still, for some reason it doesn't.

I think that is where the issue resides.

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted (edited)

I used code you provided on first page, but instead of Y-m-d i used Y-m-d\TH:i:sP 

Used Ctrl+F to search y-m-d and were replacing with Y-m-d\TH:i:sP

I had 3 lines with date.

Google sitemap Version :2.3.1

attachicon.gifgsitemap.php

I just tried this:

-downloaded and replaced gsitemap.php(the date in the file already looked like this Y-m-d\TH:i:sP. Do i still need to change date somewhere?)

-deleted old sitemap and generated new one

-renamed both sitemaps

-submited to wmt

 

I have 4001 pages submited and i still get 1119 errors

Parent tag: url
Tag: lastmod
Value: -0001-11-30T00:00:00-05:00
 
Not sure what else i can do. Gonna wait for 24 hours and see if errors will disapear but i doubt they will.
Edited by SeVi (see edit history)

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted (edited)

Why there is "-" in front of date value?

i have no idea. Just checked and some of the urls have date  2013-09-14T22:42:11-04:00 and some -0001-11-30T00:00:00-05:00.

 

Doesn't make any sense why it would generate only some of it with correct date format.

Edited by SeVi (see edit history)

Share this post

Link to post
Share on other sites
asssets Newbie

asssets

Posted
Posted

i have no idea. Just checked and some of the urls have date  2013-09-14T22:42:11-04:00 and some -0001-11-30T00:00:00-05:00.

 

Doesn't make any sense why it would generate only some of it with correct date format.

Did you check data in database?

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted

Did you check data in database?

i checked same product that has bad date in sitemap. In database it has only  date_add and date_upd and it looks fine

2013-09-28 03:40:31 2013-09-30 00:45:59  

 

Where do i find last mod of the product in DB?

Share this post

Link to post
Share on other sites
vekia Apprentice

vekia

Posted
Posted

i checked same product that has bad date in sitemap. In database it has only  date_add and date_upd and it looks fine

2013-09-28 03:40:31 2013-09-30 00:45:59  

 

Where do i find last mod of the product in DB?

 

this field name is located in ps_product_Shop table and field name is: date_upd

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted

As i understand date_upd is last modified date

 

 

this field name is located in ps_product_Shop table and field name is: date_upd

 

So if the date format in DB is correct 2013-09-30 00:45:59  , then the problem is with sitemap. Any suggestions ?

 

Share this post

Link to post
Share on other sites
guanooo Newbie

guanooo

Posted
Posted

So, I also had this problem and read all of the comments in this topic.

 

Give it a try. WT now accepts the file with no errors. But if you check the date format in the file before sending it in WT it won't be the date you've created the file. I suppose that with this file now it just ignores the date:

1. Download the attached file - gsitemap.php.

2. Delete previous sitemaps on your server.

3. Deactivate the module in BO.

4. Activate the module again.

5. Generate the sitemap(s).

6. Rename the generated file (2 files if you have multiple languages).

7. Upload the file in WT

8. Check for errors now.

 

Any success?

  • Like 1

Share this post

Link to post
Share on other sites
fishordog Newbie

fishordog

Posted
Posted (edited)

i just submitted new sitemap, so it's still pending. Will post when it's done

 

 

Update:

 

still getting errors, before it was 1119, now 1056. 

 

An invalid date was found. Please fix the date or formatting before resubmitting.
 
Parent tag: url
Tag: lastmod
Value: 1969-12-31
Edited by SeVi (see edit history)

Share this post

Link to post
Share on other sites