Helpfulcore.GeoIp – Custom Geo IP location services for Sitecore

Everybody knows that Sitecore provides services for Geo IP location. In order to be able to use that, you need to buy additional subscription. However, if you by some reason don’t want to use default Sitecore provider for Geo IP, there is a way to implement your own custom Geo IP provider.

From what I have been researching, I have found 2 services that can provide Geo IP lookup for free:

Helpfulcore.GeoIp

I tried to use both and going to share a Sitecore module that can be simply integrated as a nuget package into your Sitecore solution.
Please, find the project on github here https://github.com/vhil/Helpfulcore-GeoIp
And it is available in nuget. To install Helpfulcore.GeoIp, run the following command in the Package Manager Console

Install-Package Helpfulcore.GeoIp

Architecture and extensibility

The module is fully configuration-driven, so all dependency management happens through simple include config file.
In order to make it extendable, I implemented GeoIpLookupProvider class which inherits default Sitecore LookupProviderBase class and created a constructor for it which accepts the retriever object and data adapter object, so its simple to implement any new retrievers and then define how retrieved data has to be adapted to WhoisInformation object which Sitecore uses as a business model.
So key abstractions look like this:

the retriever:

public interface IGeoIpLocationRetriever
{
	IGeoIpResponse GetInformationByIp(string ip);
}

the adapter:

public interface IGeoIpDataAdapter
{
	WhoIsInformation Adapt(IGeoIpResponse providerObject);
}

the lookup provider:

public class GeoIpLookupProvider : LookupProviderBase
{
	//...

	public GeoIpLookupProvider(
		IGeoIpLocationRetriever geoIpLocationRetriever,
		IGeoIpDataAdapter geoIpDataAdapter,
		ILoggingService logger)
	{
		// ...
	}
}

So the principle is simple, GeoIpLookupProvider calls IGeoIpLocationRetriever to get data from whatever service you implement, then tries to adapt it with IGeoIpDataAdapter to target object of type WhoIsInformation which is used by Sitecore.

Helpfulcore.GeoIp module provides dedicated include config file

\App_Config\Include\Helpfulcore\Helpfulcore.GeoIp.config

with next configurations:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <helpfulcore>
      <geoIp>
        <!--Free Geo IP-->
        <freeGeoIpRetriever type="Helpfulcore.GeoIp.FreeGeoIpNet.FreeGeoIpNetRetriever, Helpfulcore.GeoIp">
          <param name="serviceEndpoint">http://freegeoip.net/xml/</param>
          <param name="logger" ref="helpfulcore/geoIp/loggingService"/>
        </freeGeoIpRetriever>
        <freeGeoIpAdapter type="Helpfulcore.GeoIp.FreeGeoIpNet.FreeGeoIpDataAdapter, Helpfulcore.GeoIp" />
        <!--Ip Api Com-->
        <ipApiComRetriever type="Helpfulcore.GeoIp.IpApiCom.IpApiComRetriever, Helpfulcore.GeoIp">
          <param name="serviceEndpoint">http://ip-api.com/json/</param>
          <param name="logger" ref="helpfulcore/geoIp/loggingService"/>
        </ipApiComRetriever>
        <ipApiComAdapter type="Helpfulcore.GeoIp.IpApiCom.IpApiComDataAdapter, Helpfulcore.GeoIp" />
        <!--Logger-->
        <loggingService type="Helpfulcore.Logging.LoggingService, Helpfulcore.Logging" singleInstance="true" >
          <param name="provider" ref="helpfulcore/geoIp/logProviders/debugLogFileProvider"/>
        </loggingService>
        <logProviders>
          <debugLogFileProvider type="Helpfulcore.Logging.NLog.NLogLoggingProvider, Helpfulcore.Logging.NLog" logFilePath="$(dataFolder)/logs/Helpfulcore.GeoIp.log.${date:format=yyyyMMdd}.txt" singleInstance="true">
            <param name="filePath">$(logFilePath)</param>
            <LogLevel>Debug</LogLevel>
          </debugLogFileProvider>
        </logProviders>
      </geoIp>
    </helpfulcore>
    <lookupManager>
      <patch:delete/>
    </lookupManager>
    <lookupManager defaultProvider="default">
      <providers>
        <clear/>
        <add name="default" type="Helpfulcore.GeoIp.GeoIpLookupProvider, Helpfulcore.GeoIp">
          <param name="geoIpLocationRetriever" ref="helpfulcore/geoIp/ipApiComRetriever" />
          <param name="geoIpDataAdapter" ref="helpfulcore/geoIp/ipApiComAdapter" />
          <param name="logger" ref="helpfulcore/geoIp/loggingService" />
        </add>
        <!--<add name="freeGeoIp" type="Helpfulcore.GeoIp.GeoIpLookupProvider, Helpfulcore.GeoIp">
          <param name="geoIpLocationRetriever" ref="helpfulcore/geoIp/freeGeoIpRetriever" />
          <param name="geoIpDataAdapter" ref="helpfulcore/geoIp/freeGeoIpAdapter" />
          <param name="logger" ref="helpfulcore/geoIp/loggingService" />
        </add>-->
      </providers>
    </lookupManager>
    <!--Override IP on local environment if required-->
    <!--<pipelines>
      <startTracking>
        <processor type="Helpfulcore.GeoIp.OverrideIpAddress, Helpfulcore.GeoIp" patch:before="*[@type='Sitecore.Analytics.Pipelines.StartTracking.UpdateGeoIpData, Sitecore.Analytics']">
          <param name="ipAddress">213.242.89.104</param>
          <param name="logger" ref="helpfulcore/geoIp/loggingService" />
        </processor>
      </startTracking>
    </pipelines>-->
    <experienceAnalytics>
      <api>
        <dimensions>
          <dimension id="{1879168B-AF5E-4E9C-9DAE-8B71125F2AD2}">
            <transformer type="Helpfulcore.GeoIp.DimensionKeyTransformers.RegionDimensionKeyTransformer, Helpfulcore.GeoIp"                          patch:instead="*[@type='Sitecore.ExperienceAnalytics.Api.Response.DimensionKeyTransformers.RegionDimensionKeyTransformer, Sitecore.ExperienceAnalytics.Api']">
            </transformer>
          </dimension>
        </dimensions>
      </api>
    </experienceAnalytics>
  </sitecore>
</configuration>

As you can see, there are implementations of retrievers and adapters for both mentioned services. By default it uses ip-api.com services and I will tell you why below.
There is also dedicated log file where the module logs all information so it keeps it separate.
And there is always a place to implement new retrievers and adapters for them in future.

Freegeoip.net provider

This one was first that I found and implemented. The format is simple: http://freegeoip.net/xml/{your_ip_or_hostname}
I tried to use that and in result found out that it doesn’t give very precise results, but it still works fine.
Its still included into default module config file and can be simply enabled by commenting it out.

Ip-api.com provider

Since a little further research I ran into this service. It uses very similar format: http://ip-api.com/json/{your_ip}
I can tell that I’m more or less happy with what this service returns, the stats look fine and more or less relevant.
Will attach few screenshots:

countries

regions

As you can see, there are still unrecognized regions and countries… but that is a free service 🙂

Matching regions to readable values

Hovewer, after using those services I realized there is always non-recognized regions in the experience analytics. I have figured out that both mentioned services use ISO 3166 country-region format  while default Sitecore provider uses something different.
In order to display correct regions in the XP Analytics, I had to implement this class Helpfulcore.GeoIp.Iso3166_Regions  which converts ISO codes into human-readable names, and then inject into dimensions transformation for the regions:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <experienceAnalytics>
      <api>
        <dimensions>
          <dimension id="{1879168B-AF5E-4E9C-9DAE-8B71125F2AD2}">
            <transformer type="Helpfulcore.GeoIp.DimensionKeyTransformers.RegionDimensionKeyTransformer, Helpfulcore.GeoIp"
                         patch:instead="*[@type='Sitecore.ExperienceAnalytics.Api.Response.DimensionKeyTransformers.RegionDimensionKeyTransformer, Sitecore.ExperienceAnalytics.Api']">
            </transformer>
          </dimension>
        </dimensions>
      </api>
    </experienceAnalytics>
  </sitecore>
</configuration>

Debugging on local environment

Eventually, Sitecore ignores localhost 127.0.0.1 IP and don’t call the lookup manager in order to retrieve that information. But there is a way to replace/simulate the IP being passed to the lookup manager. In default config there is commented pipeline processor which you can un-comment on your development environment in order to test and debug:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <!--Override IP on local environment if required-->
    <pipelines>
      <startTracking>
        <processor type="Helpfulcore.GeoIp.OverrideIpAddress, Helpfulcore.GeoIp" patch:before="*[@type='Sitecore.Analytics.Pipelines.StartTracking.UpdateGeoIpData, Sitecore.Analytics']">
          <param name="ipAddress">213.242.89.104</param>
          <param name="logger" ref="helpfulcore/geoIp/loggingService" />
        </processor>
      </startTracking>
    </pipelines>
  </sitecore>
</configuration>

And you can choose which IP to use directly in config.

This module nails down all the logic around Geo IP location in Sitecore in action, and can be extended further with new services if needed. Please, do pull requests if you feel like there are good services that worth sharing.
For sure, all free Geo IP services won’t give extremely precise results and I still recommend to use Sitecore subscription which as I understand uses the MaxMind databases, which give relevant information and are being regularly updated.

Enjoy.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s