Today, I post the first State of NG (SONG) video update. In this series, I will talk about the current project state and summarize general community discussions.
Video length: 10 min 48 sec
Kamil begins by thanking the community for their participation in discussions. He acknowledges concerns about using MongoDB as the project’s data store. He admits that MongoDB was chosen for its simplicity but recognizes that it may not be the best choice. The project is transitioning to PostgreSQL, a more widely known database, which should improve developer familiarity and performance. PostgreSQL allows for application-specific optimizations and provides more tooling.
Kamil addresses concerns about potential performance bottlenecks in Python. He mentions the Cython project, which can compile Python code to C. Without static typing, this can result in a 20-50% performance gain, and with static typing, even greater improvements, depending on the function’s complexity. Initial testing has shown 50%-1000% performance gain on computationally heavy methods.
- Pure Python Mode: In the context of Cython 3.0, Kamil mentions the importance of “Pure Python Mode,” which allows for writing cython modules in a Python-native way. He presents xmltodict.py as an example.
Kamil thinks about future scalability by considering the possibility of running multiple threads per user request. While not an immediate need, this approach could significantly improve performance when needed.
Kamil believes that the new database schema will operate more efficiently, though he cannot guarantee this without performance benchmarks which will come in later.
Kamil introduces the concept of client-side content encoding. By using techniques like Brotli and GZIP, the size of data transferred by users can be significantly reduced. This results in substantial performance gains for users with slower internet connections. An example changeset upload takes 5-15x less time for a person with 200kb/s internet.
Kamil addresses concerns about the project’s structure, specifically the inclusion of too much logic in models. He initially included this logic to make the transition for current contributors smoother - presenting Ruby code example. However, he acknowledges that higher-quality code is essential and mentions that this issue is being addressed as part of the database migration.
Kamil mentions that future discussions, including API 0.7 and other significant changes, will be held in the general section to encourage more public input and community involvement, as per forums governance decision.
Kamil encourages support for the development through GitHub sponsors, Patreon, or Librepay, as he is working on the project full-time. He expresses appreciation for any support, including likes and stars on the project repository.
Disclaimer: Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices.
11 posts - 4 participants
Ce sujet de discussion accompagne la publication sur https://community.openstreetmap.org/t/state-of-ng-1-postgresql-performance-project-structure/106042