Skip to content

Watch the minecraft overviewer-based Kedama map and visualize its differences among days to months(To do). WARNING: project in early development, unstable, and no backward compatibility by design.

xmoiduts/KedamaDiff

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

KedamaDiff, 毛玉Kedama差分Diff

Crawl Minecraft Overviewer-based maps and save updated map tiles, in hope to view them later [to do] 抓取 Overviewer 渲染的 Minecraft 地图并差分入库,以期日后可以查看Overviewer历史地图[待做]

WARNING: project in early development, unstable, and no backward compatibility by design. If you happen to have used it and generated some json files, try to comprehend tools/json-to-sqlite that I use to convert data file. 警告,本项目尚处于早期开发阶段,无力保证向下兼容(兼容以前的数据文件/配置文件格式)。 tools/json-to-sqlite 目录提供了作者自用的数据转换脚本。

Dev log Ep.1 | 开发日志其一

Dev log Ep.2 | 开发日志其二

Requirements 依赖

Python 3.7+

pip3

​ pytz <unify timezone|统一时区>

​ python-telegram-bot <Send status message to Telegram | 向TG发送运行状态/进度消息>

​ aiohttp <single-threaded concurrency|单线程并发抓取>

​ boto3 <S3 support|S3 对象存储支持库>

Usage 用法

Modify configs/config.py (general settings) and configs/crawl_list.py (map settings), instructions are in those files.

修改上述文件(抓取器设置,地图设置),备注见文件内注释。

cd to directory that have crawl.py ,run python3 ./crawl.py. Look at its console log output, use ctrl+c to terminate when error happens.

cdcrawl.py 所在目录,运行 python3 ./crawl.py,观察终端输出,有异常时ctrl+c关停程序

将抓取所获上传到 (Upload to) S3

在配置文件中配置S3Config()相关信息,将config.py的storage_type改为S3

运行 python3 syncer.py [1] [2]/data DB --having sqlite3 上传抓取记录数据库,已上传的数据库文件不会从本地删除。

运行 python3 syncer.py [1] [2]/images img 上传已抓取图片,已成功上传的图片会从本地文件系统删除

(Optional, unmature | 可选不成熟功能) 运行 python3 syncer.py [1] [2]/log log --having .log 上传运行日志,已上传的日志会从本地删除。

其中 [1] 为 config.py/CrawlerConfig.project_root, 默认值 .. ; [2] 为 config.py/CrawlerConfig.data_foldersW, 默认值 data-production

Script behavior 脚本会做什么

Under default configurations, crawl.py will create data folder at .. , assume crawl.py in . ;

data folder have 3 sub-folders:

  • data (crawl records)
  • images (saved images)
  • log (log)

ToDo 设想一瞥

此处放置了曾经已实现的 joiner(拼图器)所生产的图片,笔者希望藉此展示自己想做的内容。

diff出的图片(5MB) (Diff:ed world map, 5MB)

仅进行拼接的v2全图 警告,非常大80MB (World map, joined only, WARNING: very large, 80MB)

About

Watch the minecraft overviewer-based Kedama map and visualize its differences among days to months(To do). WARNING: project in early development, unstable, and no backward compatibility by design.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages