Jump to content
antoine@ultro

Maxmind GeoLite IP Database: end of free updates

Recommended Posts

As mentioned on their website, Maxmind will no longer offer upate of their City database recommended in the Prestashop documentation.

Quote

As of April 1, 2018, updated versions of the GeoLite Legacy databases will be available to redistribution license customers only, although anyone can continue to download the March 2018 GeoLite Legacy builds. On January 2, 2019, the last build will be removed from our website. GeoLite Legacy database users will need to switch to the GeoLite2 or commercial GeoIP databases and update their integrations by January 2, 2019.

3 questions:

-  does someone know another free alternative with ipv4 and ipv6 data? 

- does someone use the https://www.ip2location.com databases with success?

- can we switch from a cities database to a countries database which is less expensive? I've seen this Forge ticket PSCSX-5643, but it has been closed.

Share this post


Link to post
Share on other sites

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

Share this post


Link to post
Share on other sites

I'm also looking for IP2Location support in Prestashop. Have any developers implemented any plug-in for us to use their LITE or commercial database?

Share this post


Link to post
Share on other sites

This is known problem, but Prestashop didn't yet replayed to it: http://forge.prestashop.com/browse/PSCSX-9710

I've added all informations they need, but seems to be not a priority task for to debug ;) Perhaps somebody can up this ticket by writing there ? My report is dated January 2018 !!

The module will continue to work, but the database will be old in some months...

16 hours ago, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Not really that what I'm observing. For me accuracy is 99,99% with GeoLite.

Share this post


Link to post
Share on other sites
3 hours ago, selectshop.at said:

This is known problem, but Prestashop didn't yet replayed to it: http://forge.prestashop.com/browse/PSCSX-9710

I've added all informations they need, but seems to be not a priority task for to debug ;) Perhaps somebody can up this ticket by writing there ? My report is dated January 2018 !!

The module will continue to work, but the database will be old in some months...

Not really that what I'm observing. For me accuracy is 99,99% with GeoLite.

 

we saw much lower rates since we added geo insights tab to module/%of resolve rate.  It really depended on what part of the world your visitors were from.  here is  screen shot of 'very busy' client shop using commercial maxmind db

https://www.screencast.com/t/XuVHP0geft

we see much lower resolve rate for free db, which is moot since it's going away.....

happy localizing

Share this post


Link to post
Share on other sites

Indeed, perhaps there is a difference between countries and from where your shop is trading. Here in Europe GeoLite accuracy is 99,99%

Share this post


Link to post
Share on other sites

GeoLite has country reported as "EU" if it is not sure which country in Europe its located, same as AP. It is causing issues in some of our reporting features.

We are trying to avoid it by using IP2Location LITE which has the results by actual country. Wish we can have the IP2Location LITE solution ready for free.

Share this post


Link to post
Share on other sites

just to give a feedback about the module

I installed it on my shop running on prestashop 1.5.6 and it works just fine. It also comes with a cron job to download the bases automaticly

The support was awesome. They managed to adapt a module that uses the geolocation feature.

 

  • Like 2

Share this post


Link to post
Share on other sites

Hi @mohamed23,

Thank you for your feedback about our module :) If anyone has any suggestions to improve the addon, we'd be happy to hear them.

Share this post


Link to post
Share on other sites

I disagree above that geolitecity is as accurate as claimed unless you have built in IP resolve rate like Geo Targeting Pro

 

the big mistake people make because they do not understand is that  if you enable PS Geo Location,  non-resolved IP addresses are not allowed to shop.  PS native Geo Location does support IP country but is built to 'restrict' visitors from excluded countries. 

another problem with other solutions is that while they may localize the country via mmdb for example, when the visitor logs in their country is changed to 'registered country' which will affect specific price by country, i.e. you showed  prices for IP country but they will change if registered country is different.

Understanding how native Geo Location works (if you want to restrict shopping) is very important when considering how you implement a geo strategy.

so 75-85% resolve for geolitecity in US, and maxmind is US company....I have no confidence when declared 99% in EU.  I can tell you for sure that  is not true again because we have the stats in our module to prove otherwise.

be careful when picking a geo solution and understand how native works before implementing.

 

happy geo'ing, el

 

 

 

Share this post


Link to post
Share on other sites

What should I do here?

I currently have a GeoLiteCity.dat file in tools/geoip/.

Maxmind currently offers the lite verision with a file ending as .mmdb, GeoLite2-City.mmdb. Should I delete the old file and replace it with this?

There are 3 databases GeoLite2 City, GeoLite2 Country and GeoLite2 ASN, should I just use City?

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites
On 2018-03-14 at 3:58 PM, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

 

What does that mean, if you run this module, does it include free lifetime updates of the commercial db's. And would the user need to download the db's?

Share this post


Link to post
Share on other sites

hi el Patron your module indicates 99.99% this for visitors coming mainly from europe 

I'm with attention to the statistics of your module,
because the free maxmind file will quickly be outdated

advise everyone to buy your module

I was also waiting for you to adapt your module to the free databases of maxmind
was good for everyone, because I see the prestashop team, very apathetic regarding this subject and the end of prestashop 1.6

I'm looking at thirtybees
It's just my opinion.
 

Share this post


Link to post
Share on other sites
On 2018-10-03 at 6:59 PM, nuno.note said:

hi el Patron your module indicates 99.99% this for visitors coming mainly from europe 

I'm with attention to the statistics of your module,
because the free maxmind file will quickly be outdated

advise everyone to buy your module

I was also waiting for you to adapt your module to the free databases of maxmind
was good for everyone, because I see the prestashop team, very apathetic regarding this subject and the end of prestashop 1.6

I'm looking at thirtybees
It's just my opinion.
 

 

Thank's for the headsup on the PrestaShop fork https://thirtybees.com/

I'll definitely will be looking that up for my next project.

EDIT:

Tested it and can say it runs awesome, speed improvements at least. 2500 commits since their fork from 1.6.1.11.

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites

Have just converted the free GeoLite2 City Database to GeoLiteCity.dat (PS readable format) and uploaded into live store and it works.

Share this post


Link to post
Share on other sites

remember, native geo location logic will disallow shopping when IP does not resolve.  this is why geo targeting pro was born,  it allows you to set a country for non-resolved IP's so they can shop.  Understanding this important fact when implementing native geo location should be considered when choosing the IP lookup source.  Free has 75-85% resolve rate, a bit better in US.  Happy geo'ing,  el

Share this post


Link to post
Share on other sites
7 hours ago, toplakd said:

Have just converted the free GeoLite2 City Database to GeoLiteCity.dat (PS readable format) and uploaded into live store and it works.

Which command did you execute? 

Share this post


Link to post
Share on other sites
1 hour ago, nicoX said:

Which command did you execute? 

There is no command, we override it without geo targetig pro.  You would need  to locate the geo localization and hack that to set non-resolved IP to a default country and allow shopping.  

Share this post


Link to post
Share on other sites

If you want to use the GeoLite2 data base without transforming it into the old format you can use the module Support for MaxMind GeoLite2 Databases (MMDB) which not only updates the geolocation to the new database but also provides a cron task for keeping the database up to date.

BTW, it also allows non resolved IPs to access the shop.

Share this post


Link to post
Share on other sites
4 minutes ago, El Patron said:

There is no command, we override it without geo targetig pro.  You would need  to locate the geo localization and hack that to set non-resolved IP to a default country and allow shopping.  

I was referring to the github tool:

$ ./geolite2legacy.py -i GeoLite2-Country-CSV.zip -f geoname2fips.csv -o GeoIP.dat
Database type Country - Blocks IPv4
wrote 306385-node trie with 300679 networks (251 distinct labels) in 8 seconds

# ./geolite2legacy.py -i GeoLite2-ASN-CSV.zip -o GeoIPASNum.dat 
Database type ASN - Blocks IPv4
wrote 518484-node trie with 417952 networks (62896 distinct labels) in 15 seconds

 

 

 

Share this post


Link to post
Share on other sites

...I don't care about free maxmind or fixing so it works.....I just provide feedback of how native geolocation works and to use at your own risk.

happy geoing.....

Share this post


Link to post
Share on other sites
2 hours ago, nicoX said:

Which command did you execute? 

python geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoLiteCity.dat

 

Share this post


Link to post
Share on other sites
1 hour ago, NSN said:

If you want to use the GeoLite2 data base without transforming it into the old format you can use the module Support for MaxMind GeoLite2 Databases (MMDB) which not only updates the geolocation to the new database but also provides a cron task for keeping the database up to date.

BTW, it also allows non resolved IPs to access the shop.

Thanks, I will probably check this out, good price, and automation!

20 minutes ago, toplakd said:

python geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoLiteCity.dat

 

I did it in Windows bash. I downloaded GeoLite2 City csv zip file. I don't know what geoname2fips.csv is. There is no file like that. I tried without that as well, but ended up with:

anonX@anonx:~/geolite2legacy$ ./geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoIP.dat
Traceback (most recent call last):
  File "./geolite2legacy.py", line 32, in <module>
    import ipaddr
ImportError: No module named ipaddr
anonX@anonx:~/geolite2legacy$ ./geolite2legacy.py -i GeoLite2-City-CSV.zip -o GeoIP.dat
Traceback (most recent call last):
  File "./geolite2legacy.py", line 32, in <module>
    import ipaddr
ImportError: No module named ipaddr

 

Share this post


Link to post
Share on other sites

:)
You have to install the module which is described by ImportError: ipaddr

and after that all for which you will get error.

Share this post


Link to post
Share on other sites
3 hours ago, toplakd said:

:)
You have to install the module which is described by ImportError: ipaddr

and after that all for which you will get error.

Ok, I downloaded both ipaddr and pageoip.

Which commands do I now have to run? There is Country-CSV.zip, City-CSV.zip and ASN-CSV.zip. Which ones do I need?

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites
14 hours ago, El Patron said:

...I don't care about free maxmind or fixing so it works.....I just provide feedback of how native geolocation works and to use at your own risk.

happy geoing.....

Have one question about your Pro module.
How does it handle visitors IP's and resolving their country when shop is on cloudflare?

Share this post


Link to post
Share on other sites
4 hours ago, toplakd said:

Download database on bottom link

https://geolite.maxmind.com/download/geoip/database/GeoLite2-City-CSV.zip

Rename it to GeoLite2-City-CSV.zip (when downloaded date is added to its name)

than run.

python geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoLiteCity.dat

So not the CSV format database file?

What's the difference between City, Country and ASN?

Share this post


Link to post
Share on other sites

Nico. Read the post again. The link is to CSV Database which is than with command converted to old legacy format.

For the info about base difference check maxmind.com

Edited by toplakd (see edit history)
  • Like 1

Share this post


Link to post
Share on other sites
6 hours ago, toplakd said:

Nico. Read the post again. The link is to CSV Database which is than with command converted to old legacy format.

For the info about base difference check maxmind.com

The GeoLite2 City database returns location data down to the postal code level when available, the GeoLite2 Country database just returns the continent and country, and the GeoLite ASN returns information about who is operating the IP address.

Share this post


Link to post
Share on other sites
On 1/30/2019 at 1:56 PM, toplakd said:

Nico. Read the post again. The link is to CSV Database which is than with command converted to old legacy format.

For the info about base difference check maxmind.com

Do I need to do anything else when owerwriting the .dat file? Disable/enable the geolocation, clear the cache? 

Some customer complain they can't access my site from a specific location and country. By getting a 503 server overloaded status. Is this related to geolocation? 

Share this post


Link to post
Share on other sites
On 1/29/2019 at 4:28 PM, NSN said:

If you want to use the GeoLite2 data base without transforming it into the old format you can use the module Support for MaxMind GeoLite2 Databases (MMDB) which not only updates the geolocation to the new database but also provides a cron task for keeping the database up to date.

BTW, it also allows non resolved IPs to access the shop.

How does it allow non resolved IPs? With a default country you select? 

Share this post


Link to post
Share on other sites

I haven't exactly clarified it, but it should use your shops default you can select in the admin.

Share this post


Link to post
Share on other sites
13 minutes ago, NSN said:

I haven't exactly clarified it, but it should use your shops default you can select in the admin.

It should, or it will? Have you tested it? 

Share this post


Link to post
Share on other sites

I tested the new db. Seems to be issues with ipv6. Both my data plan and ISP are from the same county. ISP uses ipv6 and I can't reach the site. Only with ipv4 which is in my data plan. 

GeoLite2 supports ipv6? Is this a PrestaShop limitation? 

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites
vor 3 Stunden schrieb nicoX:

It should, or it will? Have you tested it? 

I'm not a programmer nor such a geek that I now how to test it. I can just rely on the developer who told me that the non resolved IPs are granted access.
I thought it would be the default as it is set in the admin and I have no idea what else should be used if the IP does not have a link to any country, but if you know how to check tell me how and I will try.

Share this post


Link to post
Share on other sites

Sorry mate I'm no geek myself. But I will ask the developer as well. 

Share this post


Link to post
Share on other sites

One thing I know is that the module uses the same database (exactly the same copy) as my matomo installation and it sometimes happens that I come across a user that has no country flag and says "country unknown".

Just happened:
matomo.png.886c72745d92f38313168fcd841ca186.png

Share this post


Link to post
Share on other sites
On 3/14/2018 at 2:58 PM, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

How is the module updating the databases? And which databas are in use, and why two? 

Share this post


Link to post
Share on other sites

@El Patron Did you read my question?

I'm asking everyone, I disabled geolocation and now my TTFB has got an huge improvement. Site loads within a half second, while when geolocation was active it took 1-2 seconds. 

Anyone else tested this and expirenced similar results?

If geolocation is adding a delay, which I think it is because it needs time to figure out the visitors location, I need another solution as web browser location or anything else. 

Share this post


Link to post
Share on other sites

I can confirm after testing with my other site that geolocation is having a huge impact on the TTFB. 

My sites now load within some ms with geolocation disabled. I can understand now since geolocation uses a database and queries the connection, adding extra load and delaying the initial connection. 

I don't know what other solutions there are to geolocation, any way for me it's not that important. 

Share this post


Link to post
Share on other sites

Hi Nico

Slow in ps 1.6??

Enable geolocation core or module??

Slow only ms not big slow 

Share this post


Link to post
Share on other sites
On 2/2/2019 at 9:34 PM, nuno.note said:

Hi Nico

Slow in ps 1.6??

Enable geolocation core or module??

Slow only ms not big slow 

I enabled core in 1.6.1.x. But as I said it has a noticable impact on the TTFB connection. I disabled it. I will use geolocation API built in web browsers HTML5 instead.

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites
Am 1.2.2019 um 8:10 PM schrieb nicoX:

How does it allow non resolved IPs? With a default country you select? 

Update to your question.

I just received the preview of an updated Version which has now the function of setting a default country by your choice for non resolved IPs.

 

Share this post


Link to post
Share on other sites

the important thing about geolocation is
- detect the country or region
(obviously that the accurate country is the most real)
- in case of not detecting, how to proceed
(the perfect thing is to consider a country by default, this country should be the country where most visitors come from)
- after login
what should be considered, cookie country or adress county

if we verify the module geolocation pro, consider these scenarios

a customer that accesses the store sees the prices and buys the products with those same prices.

do not forget that some payment modules when returning to the store the successful payment come from different ips of the customer, and will certainly create purchases with wrong prices

Attention this is not to advertise the module, I hope that the module of prestablog also consider what I said, because as it supports the new free databases of maxmind is a good alternative

I'm actually considering buying for testing.

Share this post


Link to post
Share on other sites
On 2/4/2019 at 1:25 AM, NSN said:

Update to your question.

I just received the preview of an updated Version which has now the function of setting a default country by your choice for non resolved IPs.

 

It's because I asked for it.

I spoke with a Maxmind agent and he said the CSV databases are much slower than the binaries. This module uses the binaries, so I will go for it.

@El Patron is quite unresponsive so I couldn't show any interest in a module where the dev has a hard time answering.

Share this post


Link to post
Share on other sites
vor 6 Stunden schrieb nicoX:

It's because I asked for it.

I spoke with a Maxmind agent and he said the CSV databases are much slower than the binaries. This module uses the binaries, so I will go for it.

I too think the module is the right way to go. Not only because of the original database instead of a converted one, but also because of the possibility to force a country of choice.
Another good point that you can use the free version of maxmind's databases which I think are more than enough for most users.
The updated version (1.2.0) is already available on addons, as I saw recently.

Share this post


Link to post
Share on other sites
36 minutes ago, NSN said:

I too think the module is the right way to go. Not only because of the original database instead of a converted one, but also because of the possibility to force a country of choice.
Another good point that you can use the free version of maxmind's databases which I think are more than enough for most users.
The updated version (1.2.0) is already available on addons, as I saw recently.

I have also been looking at https://www.prestaheroes.com/en-us/modules/geo-localization/geo-targeting-pro-price-by-country

MaxMind GeoIP2 Country 99.8% accurate on a country level

MaxMind GeoIP2 Precision Country 

I think those two are commercial databases. But it lets you select the free db as well. 

Share this post


Link to post
Share on other sites
vor einer Stunde schrieb nicoX:

I have also been looking at https://www.prestaheroes.com/en-us/modules/geo-localization/geo-targeting-pro-price-by-country

MaxMind GeoIP2 Country 99.8% accurate on a country level

MaxMind GeoIP2 Precision Country 

I think those two are commercial databases. But it lets you select the free db as well. 

I did look at it too. To be honest, I did look at it for a long time because the module looks to be very functional and well developed, but I could not make up my mind to buy it. The various functions a very useful, but nothing I need in particular so that I would be willing to pay so much for it (the module is one part but you also need the database, so the total amount increases). If you then buy the database only once, the idea of the various functions and accuracy is gone, and if you pay for regularly updates it costs quite a lot. (From the module description and documentation it's not visible that the new free database is also supported)
If it is worth paying so much for all together has everybody to decide for himself, but for me it wasn't. Sure the accuracy of the pro database is great, but I have no problem if occasionally someone from Canada gets USD instead of CAD, or a European sees JPY instead of EUR, so the free one is more than enough for me.

In a nutshell, if you need a lot of functionality and high accuracy you should go with the one from PrestaHeroes and than also pay for regularly updates of the pro database.
If you just want to upgrade the basic function of native geolocation to work with the new free database and extend the functionality to deal with non resolved IPs I think the one from PrestaBlog is more than enough.

Share this post


Link to post
Share on other sites

I'm little confused with PS Localization> Localization> Configuration

  • Set language from browser (Set browser language as default language)
  • Set default country from browser language (Set country corresponding to browser language) YES/NO

Is this some sort of geolocation acting on the browser language. Is this good to use and will it conflict with geolocation if both are enabled?

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites

if you use geolocation, with undetected country override (country by default), you should not use the default country in browser

Share this post


Link to post
Share on other sites
Posted (edited)
On 2/19/2019 at 12:28 PM, nuno.note said:

if you use geolocation, with undetected country override (country by default), you should not use the default country in browser

What about the Set language from browser?

If I use Geolocation should this be on or off?

I'm just interested about the currency. We only use English language, but would like EURO countries to see EURO, UK to see Pounds.

Edited by nicoX (see edit history)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Cookies ensure the smooth running of our services. Using these, you accept the use of cookies. Learn More