Datafile not found, datafile generation failed in confusable_homoglyphs/categories.py

user_2740177 0
Viewed 521

I'm setting up a django project on an apache server. It is running fine on my PC under using manage.py runsslserver.

When I load onto the apache server I can run manage.py migrate and collectstatic and a number of setup programs.

BUT when I try to enter the webpage I get error message "Datafile not found, datafile generation failed" with this traceback:

Environment:


Request Method: GET
Request URL: https://ekim.hexiawebservices.co.uk/

Django Version: 1.11.5
Python Version: 2.7.12
Installed Applications:
['django.contrib.admin',
 'registration',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'social_django',
 'djangosecure',
 'sslserver',
 'ekim']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "/var/www/ekim/env/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/var/www/ekim/env/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response
  172.         resolver_match = resolver.resolve(request.path_info)

File "/var/www/ekim/env/lib/python2.7/site-packages/django/urls/resolvers.py" in resolve
  362.             for pattern in self.url_patterns:

File "/var/www/ekim/env/lib/python2.7/site-packages/django/utils/functional.py" in __get__
  35.         res = instance.__dict__[self.name] = self.func(instance)

File "/var/www/ekim/env/lib/python2.7/site-packages/django/urls/resolvers.py" in url_patterns
  405.         patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)

File "/var/www/ekim/env/lib/python2.7/site-packages/django/utils/functional.py" in __get__
  35.         res = instance.__dict__[self.name] = self.func(instance)

File "/var/www/ekim/env/lib/python2.7/site-packages/django/urls/resolvers.py" in urlconf_module
  398.             return import_module(self.urlconf_name)

File "/usr/lib/python2.7/importlib/__init__.py" in import_module
  37.     __import__(name)

File "/var/www/ekim/ekim/urls.py" in <module>
  30.     url(r'^accounts/', include('registration.backends.hmac.urls')),

File "/var/www/ekim/env/lib/python2.7/site-packages/django/conf/urls/__init__.py" in include
  50.         urlconf_module = import_module(urlconf_module)

File "/usr/lib/python2.7/importlib/__init__.py" in import_module
  37.     __import__(name)

File "/var/www/ekim/env/lib/python2.7/site-packages/registration/backends/hmac/urls.py" in <module>
  10. from . import views

File "/var/www/ekim/env/lib/python2.7/site-packages/registration/backends/hmac/views.py" in <module>
  15. from registration.views import ActivationView as BaseActivationView

File "/var/www/ekim/env/lib/python2.7/site-packages/registration/views.py" in <module>
  12. from registration.forms import RegistrationForm

File "/var/www/ekim/env/lib/python2.7/site-packages/registration/forms.py" in <module>
  19. from . import validators

File "/var/www/ekim/env/lib/python2.7/site-packages/registration/validators.py" in <module>
  10. from confusable_homoglyphs import confusables

File "/var/www/ekim/env/lib/python2.7/site-packages/confusable_homoglyphs/confusables.py" in <module>
  7. from .categories import unique_aliases, alias

File "/var/www/ekim/env/lib/python2.7/site-packages/confusable_homoglyphs/categories.py" in <module>
  147.         raise Exception('Datafile not found, datafile generation failed!')

Exception Type: Exception at /
Exception Value: Datafile not found, datafile generation failed!

I have been trying to sort this for 3 hours and cannot work it out. Any help greatly appreciated.

django python
2 Answers

I have the same problem, and after doing some research I've discovered the source of the problem: the registration library requires confusable-homoglyphs, and confusable-homoglyphs looks for two files: categories.json and confusables.txt. If it doesn't find those files, it tries to download two files from unicode.org - Scripts.txt and confusables.txt - and generate the two json files from those. If you download those files too often, unicode.org will throttle your bandwidth and possibly even blacklist your address.

confusable-homoglyphs is hard-wired to put the files in the working directory returned by os.getcwd(), so I downloaded those files on a different computer and put them in my working directory. The Django migration was successful. I hope this works for you too!

user_33840470

(I couldn't comment on the question because I don't have enough karma.)

You encountered this error because you rely on a library I wrote. It downloads some files from unicode.org whenever it gets loaded for the first time. Unfortunately I didn't expect a few use cases, for instance deploying apps with my lib as a dependency on Platforms as a Service, where dependencies would frequently get downloaded, which would also cause my lib to hit unicode.org much more frequently than I had expected.

Fortunately someone submitted a patch to the lib, fixing this issue for good. Please upgrade to confusable_homoglyphs>=3.0.0. :)

user_43593690
Related