Commit Graph

246 Commits

Author SHA1 Message Date
Fabian Schlenz deb5e316cb Kotlin code is now compileable. 2017-12-07 06:12:51 +01:00
Fabian Schlenz 3309ae1338 Complete conversion. 2017-12-07 08:00:18 +01:00
Fabian Schlenz 166df82ac1 First tries. 2017-12-06 11:14:11 +01:00
Fabian Schlenz 9693371a3d Changed the order of code in CommandLineController to allow login when no database is present yet. Duh. 2017-12-06 06:09:45 +01:00
Fabian Schlenz 5328df65b3 Merge branch 'feature-channels-and-supergroups'. Closes #21. 2017-12-05 19:10:18 +01:00
Fabian Schlenz fff3483e63 Fixed stats in the HTML export for channels. 2017-12-02 15:05:04 +01:00
Fabian Schlenz 99574c6e6e Split --with-channels-and-supergroups into seperate switches --with-channels and --with-supergroups. 2017-12-01 21:17:13 +01:00
Fabian Schlenz b8d9e2a1b6 Some improvements in the output. 2017-12-01 14:13:58 +01:00
Fabian Schlenz 459de54ae7 Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2017-11-30 13:35:09 +01:00
Fabian Schlenz 7bf22f4692 Implemented backing up of supergroups and channels added. Finally. Yaaaay. Be aware: There will possibly be a lot of bugs! 2017-11-30 13:33:36 +01:00
Hypfer 0de09c45dc Fix #41
I don't know why that one file is broken and nor do I care.
This could use some debug logging. Well. Whatever.
2017-03-20 20:26:40 +01:00
Fabian Schlenz 14d80a70fe '--stats' now also counts chats and users in the DB. 2017-02-26 13:20:36 +01:00
Fabian Schlenz 1a72ab8735 Expanded `--stats` to contain information about api_levels of messages in the db. 2017-02-23 06:38:43 +01:00
Fabian Schlenz 6b44a6c569 Logging now with millisecond precision. 2017-02-23 06:22:36 +01:00
Fabian Schlenz c945f05c3d Added '--stats'. 2017-02-22 18:37:28 +01:00
Fabian Schlenz 6d772a3be1 UserManager is now a Singleton as well. 2017-02-22 06:57:58 +01:00
Fabian Schlenz 42112d7607 Updated the year in the version string. 2017-02-22 06:54:01 +01:00
Fabian Schlenz a2224b326d Database is now a singleton. 2017-02-22 06:53:51 +01:00
Fabian Schlenz 92fb628b92 Some refactoring to split long methods into smaller methods. 2017-02-22 06:43:41 +01:00
Fabian Schlenz 1d8724ecb7 Catching TimeoutExceptions during mediaDownload() should now skip the file. 2017-02-21 13:41:55 +01:00
Fabian Schlenz 6738e20fe4 Added timeout detection to media downloads as well. 2017-02-18 12:56:56 +01:00
Fabian Schlenz e0668926c9 More exception outputting. 2017-02-17 06:34:20 +01:00
Fabian Schlenz 715b86b09b Tried skipping files in case of a timeout instead of just retrying. Then noticed that the timeout couldn't actually be thrown. Added logging for that case to investigate. 2017-02-15 08:27:11 +01:00
Fabian Schlenz a8325735a3 Fixed NPE from DocumentFileManager.isSticker. #30. 2017-02-14 22:10:24 +01:00
Fabian Schlenz fb2db5b639 Revert e9df4eb, experimental randomization of number of messages to download, because it didn't change anything (except for slowing the download even more). 2017-02-14 06:46:40 +01:00
Fabian Schlenz e9df4ebdb4 Experiment: Randomize the number of requested messages. 2017-02-14 06:45:38 +01:00
Fabian Schlenz e0afcc2063 Merge pull request #28 from ffflorian/patch-01
Added title for chat times chart
2016-11-25 21:44:14 +01:00
Fabian Schlenz 4ef11a1953 Added even more debug output to HTMLExporter. References #33, #16. 2016-11-25 21:43:33 +01:00
Fabian Schlenz 4b92d63d2e Revert "Testing putting multiple GetMessages requests into one RpcQuery. References #35."
This reverts commit 0249374384. Closes #35.
2016-11-25 21:24:43 +01:00
Fabian Schlenz 0249374384 Testing putting multiple GetMessages requests into one RpcQuery. References #35. 2016-11-25 21:23:45 +01:00
Fabian Schlenz 6a90f2ef5c Size of the Batch of IDs for getMessages() is now configurable. Also, upped the limit from 100 to the current 200. 2016-10-26 21:21:26 +02:00
Fabian Schlenz 3d213e9780 obeyFloodWaitException() now also has a silent mode for repeated FLOOD_WAITs. Also, DownloadController now also checks if the amount of returned messages equals the number of requested messages. 2016-10-26 21:20:46 +02:00
Fabian Schlenz e32924fc00 CommandLineController.show_error() now also sends the error to the logger. 2016-10-26 21:19:05 +02:00
Florian Keller 94c1a413ee
Added title for chat times chart 2016-10-06 22:05:41 +02:00
Fabian Schlenz f73a00f9cc Added --anonymize to anonymize sensitive data before sending in the output of this app. Closes #20. 2016-10-05 16:08:22 +02:00
Fabian Schlenz 42da94d290 Stickers are now shown in the HTML export. Closes #24. 2016-10-05 13:47:25 +02:00
Fabian Schlenz 3a615f94af Moved the stickers into the files directory of the individual accounts. Instead of downloading the stickers again, they will be copied from the old location. Closes #25. 2016-10-05 13:41:11 +02:00
Fabian Schlenz cb36fb29a6 Added some more trace output for HTML export. Could be helpful for example for solving ticket #16. 2016-10-05 12:37:44 +02:00
Fabian Schlenz 89073d128a Typo for 42fabf7 corrected. 2016-09-21 21:12:08 +02:00
Fabian Schlenz 45adc77d5e Added special handling for plain text files without given extension: Will now use .txt instead of .plain. 2016-09-21 21:07:11 +02:00
Fabian Schlenz 42fabf7b1c Centralized creation of writers in the HTML exporter and added some logging. This could help with #16. 2016-09-21 21:05:53 +02:00
Fabian Schlenz 2ccb0cea9f Replaced Thread.sleep by TimeUnit.SECONDS.sleep and colleagues. Doesn't change the way the code works and still uses Thread.sleep internally, but now the code is much easier to read. 2016-09-20 18:44:03 +02:00
Fabian Schlenz 517e5b68ed Experimental: Explicitly save HTML files in the exporter as UTF-8. 2016-09-20 06:36:13 +02:00
Fabian Schlenz 81ac38fbc8 Stop looking for an account after a matching account is found. 2016-09-20 06:35:25 +02:00
Fabian Schlenz cc7b2f09af Removing double quotes from document file extensions. Fixes #12. 2016-09-19 06:54:36 +02:00
Fabian Schlenz 5b2107b7ae Charts are now visible even if the html files are not served via http or https. Fixes #13. 2016-09-19 06:52:08 +02:00
Fabian Schlenz ef89185439 Added switch to output some encoding values. 2016-09-18 11:33:27 +02:00
Fabian Schlenz fa864f37b7 Increased the delay between getMessages from 0.1 to 0.4 seconds in a hope to prevent FLOOD_WAIT exceptions. 2016-09-16 06:41:06 +02:00
Fabian Schlenz 4d7e46caa1 Removed the extra waiting time when obeying FLOOD_WAIT exceptions. 2016-09-15 06:37:42 +02:00
Fabian Schlenz 5b9de9d7cc The update check now also outputs the new version's description. 2016-09-15 06:36:39 +02:00
Fabian Schlenz 3db48864bc Trying to rename downloaded files more than once. This could help against aggressive anti virus tools. 2016-09-15 06:23:10 +02:00
Fabian Schlenz 7e26ff0849 Added support for accounts having more than 1 million messages. 2016-09-05 06:47:01 +02:00
Fabian Schlenz 5a9cd6bd80 Apparently there can be TLDocuments without any TLDocumentAttributes. This interrupted the message download with an NPE. Fixes #10. 2016-09-05 06:14:52 +02:00
Fabian Schlenz 11b5dc66e5 Version check now sees 1.0.4-pre.1 as older than 1.0.4, but 1.0.4-abd332 as newer. 2016-09-02 16:38:00 +02:00
Fabian Schlenz 71fb63443b Added a test feature reachable by using `--test 1` to test entries in a local cache4.db file for compatibility. 2016-09-02 06:50:49 +02:00
Fabian Schlenz ed442765ef Slightly modified log levels in DownloadManager. 2016-08-31 07:05:00 +02:00
Fabian Schlenz 8f5a9a9ad5 Renamed --debug-telegram to --trace-telegram and added --trace, while --debug now shows only log entries up to DEBUG level. 2016-08-31 07:04:12 +02:00
Fabian Schlenz dfd714e481 Added command line option --no-media to skip downloading media files. 2016-08-31 07:03:49 +02:00
Fabian Schlenz 34cca18009 The check for the last message ID should ignore channels, because they have their own numbering system which could result in numbers larger than your own message IDs which leads to downloads of lots of empty messages which then block further backups. 2016-08-31 06:55:22 +02:00
Fabian Schlenz 5291b35f63 Added version checking code. Complete with unit tests. Oh yeah. :D 2016-08-29 07:01:32 +02:00
Fabian Schlenz 3e64b336a8 Only create a database backup if there isn't one already. This prevents overwriting a perfectly fine backup with a more or less broken one in case a database update fails. 2016-07-27 06:11:50 +02:00
Fabian Schlenz eb9715cb09 * Explicitly close ResultSet in DatabaseUpdate process to prevent not being able to update the Database due to it being locked. Closes #6.
* Don't create backup of non-existing database if we're creating a fresh one.
* Changed the style of HTML exports slightly.
2016-07-26 21:24:21 +02:00
Fabian Schlenz fad822d53d Removed the old StatsExporter since the stats are now included in HTMLExporter's output. 2016-07-26 18:15:25 +02:00
Fabian Schlenz cbdb5dfcb9 Massively better exports. Stylesheets and Stats added. 2016-07-26 18:10:30 +02:00
Fabian Schlenz 68f8841d12 HTMLExporter: Moved from Google charts to Highcharts; implemented the heatmap. 2016-07-26 06:42:48 +02:00
Fabian Schlenz 24f8320a42 Fixed the login process when two-factor authorization is used. 2016-07-26 05:59:58 +02:00
Fabian Schlenz 458894a8bf Improvements in account handling; detection for a mismatch between selected account and used account authorization. 2016-07-26 05:54:11 +02:00
Fabian Schlenz 8a266dedfe If your auth data is invalid or missing, you will be prompted to re-login to telegram. 2016-07-26 05:49:20 +02:00
Fabian Schlenz 7b28f8b7d0 Small code fixes / improvements. 2016-07-25 18:44:37 +02:00
Fabian Schlenz 75566ef53a WIP: Changes to the exporter. Stats are included in HTML output; Heatmap added. 2016-07-25 18:21:52 +02:00
Fabian Schlenz 627cef0ae2 Added some more debugging to UserManager. 2016-07-25 16:24:28 +02:00
Fabian Schlenz 77bfcfb849 Added a .gitkeep to build/resources/main in order to have git automatically create that folder on checkout / clone. 2016-07-21 06:36:23 +02:00
Fabian Schlenz 1481e5625e Completely reworked logging.
* Using slf4j and logback for logging.
* Disabled all logging output from kotlogram by default.
2016-07-20 06:37:24 +02:00
Fabian Schlenz a43b81ee91 Rewritten FLOOD_WAIT detection code. 2016-07-15 22:22:58 +02:00
Fabian Schlenz d339472ba8 Getting an error 500 while downloading files now skips the current file. If it happens two times in a row, we quit. 2016-07-15 22:22:18 +02:00
Fabian Schlenz 3546127690 Photos also have a dcId. Use it to prevent unnecessary FILE_MIGRATE_* stuff. 2016-07-15 07:04:58 +02:00
Fabian Schlenz 7181ec69d3 Using a special downloaderClient for fiel downloads. 2016-07-15 07:04:45 +02:00
Fabian Schlenz f2b6bc6e16 Fix NPE at Database.java line 199. 2016-07-15 07:01:09 +02:00
Fabian Schlenz 051c0a31af Cleanup 2016-07-15 06:35:06 +02:00
Fabian Schlenz f460d2307a Caching partially downloaded media data. 2016-07-14 19:47:10 +02:00
Fabian Schlenz fc7d3fdcbc Updated Kotlogram to the current HEAD. This also added a small database change and the need to update database entries for messages with media that were downloaded with api layer 51. 2016-07-14 19:43:41 +02:00
Fabian Schlenz b4c50a0163 Preparations for caching downloaded files. 2016-07-14 06:43:44 +02:00
Fabian Schlenz ffddfe68d1 More Logging. 2016-07-14 06:21:58 +02:00
Fabian Schlenz 61b5ba7676 Always display exceptions. 2016-07-14 06:15:21 +02:00
Fabian Schlenz 0b95b42607 Moved Dabase changes into their own class. 2016-07-13 06:56:00 +02:00
Fabian Schlenz ac67818d28 App now always exits. 2016-07-13 06:22:11 +02:00
Fabian Schlenz c5c692d61d Useful logging when called with --debug. The old --debug is now --debug-telegram. 2016-07-13 06:20:51 +02:00
Fabian Schlenz 95a02e89c3 Always show creator info when running. 2016-07-13 06:07:37 +02:00
Fabian Schlenz ee16af7d3c Version now automatically gets set by gradle. 2016-07-13 06:06:26 +02:00
Fabian Schlenz fefebc9cd6 Cleaned up CommandLineOptions. 2016-07-11 18:17:42 +02:00
Fabian Schlenz 6499aed3c1 CommandLineOptions is now static to make usage easier. 2016-07-11 18:11:12 +02:00
Fabian Schlenz 71150a36d2 Don't die with NPEs if a MediaFileManager tries to download an Empty media thing. 2016-07-11 06:34:16 +02:00
Fabian Schlenz 468c5517dd Supergroups have their own message IDs. Load more than the 1 last dialog and take the biggest ID. 2016-07-11 06:31:06 +02:00
Fabian Schlenz 7e91ec092e Stop execution if an error happens during database upgrades. 2016-07-11 06:25:19 +02:00
Fabian Schlenz 05dea702c3 Randomly choose file download block sizes in the hope of getting rid of FLOOD_WARN errors. 2016-07-11 06:21:34 +02:00
Fabian Schlenz 99b25c7272 Reduced the file download chunk size from 10 to 1 MByte. 2016-07-09 12:25:27 +02:00
Fabian Schlenz 41f8fc80c5 WIP: Added StatsExporter. 2016-07-09 12:19:42 +02:00
Fabian Schlenz 8b361d5547 More and nicer exports. Mustache is nice. 2016-07-08 14:21:11 +02:00
Fabian Schlenz 4e227e55c8 Added a method to create database backups before doing huge database update stuff. 2016-07-08 06:58:58 +02:00
Fabian Schlenz db52b8da47 Big database update. Removed old fields, renamed others to make more sense, added a few new ones. 2016-07-08 06:53:14 +02:00
Fabian Schlenz 2e43ad90a6 Completely reworked the file download part and made it much, much more object oriented. 2016-07-07 17:10:00 +02:00
Fabian Schlenz ecbca30a10 WIP: Using mustache as template engine for HTML exporter. 2016-07-07 06:40:00 +02:00
Fabian Schlenz c62e6db3d3 TelegramUpdateHandler now only displays debug information if you set .debug to true. 2016-07-07 06:30:00 +02:00
Fabian Schlenz 5253145024 Only run the code in TelegramUpdateHandler if user and db are set. 2016-07-07 06:24:07 +02:00
Fabian Schlenz 2bee99518b Bumped the version to 1.0.3. 2016-07-07 06:18:58 +02:00
Fabian Schlenz 515efb2fde WIP: Added Daemon-mode to keep the app running and save new messages in real time. 2016-07-06 18:07:04 +02:00
Fabian Schlenz 1c92f02422 Added a new database to track runs of this tool in order to be able to monitor them later on (Nagios, Icinga and so on). 2016-07-06 13:58:25 +02:00
Fabian Schlenz 0d431beba6 Moved FLOOD_WAIT exception handling to Utils class. 2016-07-06 08:30:23 +02:00
Fabian Schlenz b3913ac123 Moved login code to it's own method; using System.console() (if possible) to let the user enter his password without it being echoed to the console. 2016-07-06 08:08:20 +02:00
Fabian Schlenz 2cb1d717b3 Bumped the version to 1.0.2. 2016-07-06 07:52:34 +02:00
Fabian Schlenz 799bad9755 FLOOD_WAIT-Protection now adds a few minutes of waiting time to give the API time to relax. Also increased the delay between file downloads to 1.5 seconds. 2016-07-06 07:52:20 +02:00
Fabian Schlenz 30618b4e3e Bumped the version to 1.0.1. 2016-07-05 17:40:18 +02:00
Fabian Schlenz 07aec6a94d Added GPLv3 headers to all the source files. 2016-07-05 15:32:54 +02:00
Fabian Schlenz 989e2e1d5c Added a short text to inform the user of how to exit the program if it doesn't do that by itself. 2016-07-05 15:32:20 +02:00
Fabian Schlenz 140dfdfabc Better error message for max_database_id>max_message_id. 2016-07-05 15:22:32 +02:00
Fabian Schlenz f5a478a454 HTMLExporter now also outputs the time of messages. 2016-07-05 15:22:08 +02:00
Fabian Schlenz 84984ddc20 Prepared to catch TimeoutExceptions and restart the download, if necessary. 2016-07-05 14:20:32 +02:00
Fabian Schlenz d2795cc7ab The code is now under GPLv3 license. 2016-07-05 14:03:46 +02:00
Fabian Schlenz bfdd70933d Added FLOOD_WAIT detection. 2016-07-05 13:31:45 +02:00
Fabian Schlenz bad165e97b Added support for downloading Geolocation maps. 2016-07-05 12:56:14 +02:00
Fabian Schlenz c8e33fa319 Added beginnings of an HTMLExporter. 2016-07-05 12:55:25 +02:00
Fabian Schlenz cf6ffc4c60 Fix for NPEs if the target directory does not exist. 2016-07-05 07:56:18 +02:00
Fabian Schlenz b3018ad10e Compatibility to JAVA 1.7 2016-07-05 07:55:53 +02:00
Fabian Schlenz 523ee51d22 Added a feature to display the version. 2016-07-04 15:45:45 +02:00
Fabian Schlenz e03f96363f Files get saved to the user dir by default, but you can change this value by using '--target'. 2016-07-04 15:37:23 +02:00
Fabian Schlenz 07ba594a15 Added automatic checks for missing messages in the database complete with download of these mesages. 2016-07-04 13:33:17 +02:00
Fabian Schlenz 29b963ec25 We only need the most recent dialog in order to get the top message id. 2016-07-04 11:04:35 +02:00
Fabian Schlenz 3b01f47f38 First database schema update. 2016-07-04 10:19:55 +02:00
Fabian Schlenz 83664b8fc0 Better support for empty media types. 2016-07-04 10:19:36 +02:00
Fabian Schlenz 8c0bb8aa83 Removed main.iml 2016-07-04 09:36:03 +02:00
Fabian Schlenz 293e77636e Delete files with too less bytes. 2016-07-04 09:35:40 +02:00
Fabian Schlenz 45e3e22ef3 Users and chats are saved to the database as well; DB schema changed requiring you to delete your database.sqlite at this point; old 'good' messages will not be overwritten by newer 'empty' messages. 2016-07-04 09:35:12 +02:00
Fabian Schlenz 7eed1f6bb1 Configurable delay between Message downloads. 2016-07-04 09:33:42 +02:00
Fabian Schlenz 95e74c58eb Better command line output during media download. 2016-07-04 09:32:53 +02:00
Fabian Schlenz 28965277e7 Added support for audio media. 2016-07-04 09:32:30 +02:00
Fabian Schlenz a2d3dadf7b File size-aware file downloading. Fixes #1. 2016-07-02 19:43:01 +02:00
Fabian Schlenz d75d409f87 If there is just one account, use it if possible. 2016-07-02 19:42:19 +02:00
Fabian Schlenz ba7e76087a Beginnings of a Swing GUI. 2016-07-02 14:12:57 +02:00
Fabian Schlenz c0bc3cdd2d You could call this version a first beta version. 2016-07-02 10:55:20 +02:00
Fabian Schlenz 265f3e854a This looks more or less usable... 2016-07-01 11:25:44 +02:00
Fabian Schlenz d60a56711a Download-stuff. Files. 2016-07-01 07:40:08 +02:00
Fabian Schlenz 29d5fc37d0 Commandline-Stuff, Database-stuff. 2016-06-30 10:54:40 +02:00
Fabian Schlenz 1c3d10befb Kotlogram works, login is working. 2016-06-30 07:47:37 +02:00
Fabian Schlenz a7caf248a3 removes basically all code. Fresh start. 2016-06-29 11:01:26 +02:00
Fabian Schlenz 3f5fd7787c Removed libraries. 2016-06-29 11:00:29 +02:00
Fabian Schlenz 53d2b1674f First commit: Just a collection of library sources from Github. Compiles, but doesn't work. 2016-06-29 10:59:33 +02:00