Project Sencomment, HU Berlin
Oct 2011, 12 months
Position
Python Developer
Responsibilities
Creating design specifications based on requirements for new and existing products
Work within a lean software process to design and develop new product features
Acquiring skills enabling the creation of scalable functionality with Python
Documenting new features
Writing unit tests
Writing functional tests
Resolve errors and performance issues
Share knowledge and support
Constant evaluation, optimization and enhancement of existing functionalities
Leading the backend team
Link
https://fis.hu-berlin.de/converis/portal/Project/403360821?auxfun=&lang=de_DE
Technologies
linux
python
flake8
postgresql
tornado
django
Project description
SenComment used the modern Natural Language Processing (NLP) tools, for discovering the author's social properties based on the linguistic patterns found texts.
Further this information would be used for generating narrow targeted online ads.
The system was containing of multiple components, allowing the data gathering, transformation, analysis and delivery to end ads platform
First a scrapy based web crawler collects the information from different sources and organizes them into two big groups: the text marked with gender and social properties of the author and those corresponding to the product descriptions. The texts are cleaned stored in the relational database.
At the second step the machinery analyzes the language features of gathered texts and matches them with with each other, building the distribution of typical buyer of the product: gender, age and education level.
At the third step the system generates fine grained marketing information for needed for importing it into advertising platform: target groups, product description and media.
The access to data was enabled with django based web interface.