Compare commits

...

181 Commits

Author SHA1 Message Date
Fabian Schlenz 94785357bc
Updated README.md to reflect the state of the project. 2020-04-27 18:32:37 +02:00
Fabian Schlenz 223a0fdde3 Added blacklist_extensions to skip some file types completely from being downloaded. 2018-04-20 06:54:29 +02:00
Fabian Schlenz bce5996643 Show (temporary) messages every time a FLOOD_WAIT is happening. 2018-04-20 06:53:25 +02:00
Fabian Schlenz cf806da77d Added spinners to visualize the ongoing download of bigger media files. 2018-04-20 06:52:30 +02:00
Fabian Schlenz c2b1c0625e Playing around with exporters. Added a stupidly simple CSV exporter. 2018-04-20 06:43:48 +02:00
Fabian Schlenz 3c68e6d814 Merge branch 'feature-json2' 2018-04-20 06:11:11 +02:00
Fabian Schlenz 7e2d49ef09 Added a setting max_file_size to ignore files bigger than that size in MB. 2018-04-20 06:08:17 +02:00
Fabian Schlenz 11a6318a26 JSON objects now also contain a property api_layer containing their (surprise!) api_layer. 2018-04-19 17:48:11 +02:00
Fabian Schlenz bb2a291d4f MediaFileManagers now use JSON instead of the TLMessage objects. 2018-04-17 06:36:39 +02:00
Fabian Schlenz b1e9346203 Anonymize "target dir after options". 2018-04-16 05:59:28 +02:00
Fabian Schlenz 6b5b9a669b WIP: Reworking mediafilemanagers to work with JSON. 2018-04-16 05:57:05 +02:00
Fabian Schlenz 6b9cc9533a Better status reports from the DatabaseUpdate. 2018-04-14 15:01:54 +02:00
Fabian Schlenz 4c6c049502 Removed duplicate headers printed while downloading media. 2018-04-13 06:18:20 +02:00
Fabian Schlenz 28b402d3ab deploy.sh and deploy_bet.sh will now print short git diff stats into the telegram group's message. 2018-04-13 06:17:33 +02:00
Fabian Schlenz 6d4701189b Added json columns to users and chats tables; newly incoming objects are now saved together with their JSON data. 2018-04-13 06:10:27 +02:00
Fabian Schlenz c963a8f334 deploy.sh: Go back to master after merging into stable. 2018-04-12 06:55:06 +02:00
Fabian Schlenz 2402356013 Added a DatabaseUpdate to convert the data to json. 2018-04-12 06:54:14 +02:00
Fabian Schlenz df1c90578b Disabled version check if version number contains a dash. 2018-04-12 06:47:24 +02:00
Fabian Schlenz 96d5b5c77b DownloadManager now looks at all dialogs, not just the last 100. 2018-04-12 06:46:42 +02:00
Fabian Schlenz c5f901f4a2 Small codefix to use parameter instead of fixed value. 2018-04-12 05:57:11 +02:00
Fabian Schlenz e8c28b4e72 `--limit-messages` now also affects channels and supergroups. 2018-04-12 05:56:30 +02:00
Fabian Schlenz f434482cdf Database connection will now always be closed on VM shutdown. 2018-04-12 05:55:45 +02:00
Fabian Schlenz 4d45c7f1cc Copy config.sample.ini to the right folder. 2018-04-11 06:26:05 +02:00
Fabian Schlenz 955ae42952 Use exit code 1 if quitting because of an exception. 2018-04-11 06:24:59 +02:00
Fabian Schlenz 01590b05ee Reformatted the exporter selection part of CommandLineController. 2018-04-11 06:24:24 +02:00
Fabian Schlenz b5ff4dd1a6 Removed debug output from CommandLineOptions. 2018-04-11 06:16:42 +02:00
Fabian Schlenz b199dc335f Queries in Database now go through a custom function executeQuery which throws better (read: informative) Exceptions. 2018-04-11 06:14:55 +02:00
Fabian Schlenz 1ff540977e Merge branch 'feature-rewrite' 2018-04-11 06:11:07 +02:00
Fabian Schlenz ff9163c1bb Settings: Is CLI is set, use it instead of INI. 2018-04-11 05:51:00 +02:00
Fabian Schlenz 38fce0ee5c Some more refactoring of Settings. 2018-04-11 05:50:26 +02:00
Fabian Schlenz f24e66271f Added command '--settings', which will print the currently active settings. Settings can now be marked as secret - if that is true, the settings value will only be printed if it is not the default value. 2018-04-10 06:38:28 +02:00
Fabian Schlenz a9444e7813 Rewrote CommandLineOptions: The code is now better readable and accepts parameters like `--target=/data` and accepts some short parameters like -t for --target. 2018-04-10 06:36:18 +02:00
Fabian Schlenz 069799cbaf Resorted the code in DownloadManager - no more need for downloadMessages calling _downloadMessages or downloadMedia calling _downloadMedia. 2018-04-10 06:33:15 +02:00
Fabian Schlenz 9affb47130 Reworked the obeyFloodLimit-Stuff to now use lambdas. 2018-04-10 06:26:46 +02:00
Fabian Schlenz be1cf8ba91 Fixed a bug in Utils.anonymize 2018-04-10 06:13:29 +02:00
Fabian Schlenz 5c466131d3 Rewritten Settings. 2018-04-10 06:12:02 +02:00
Fabian Schlenz e5da546386 CommandLineOptions will now insert booleans into values with an value of "true" as well. 2018-04-10 06:11:01 +02:00
Fabian Schlenz ec4097e777 The code is now compiling, but still largely untested. So it's still kinda WIP. 2018-04-09 06:07:53 +02:00
Fabian Schlenz 77efde1136 WIP: Still having fun with git stash. -.- 2018-04-06 06:21:39 +02:00
Fabian Schlenz 253b334fc3 More rewriting. Also lots of fun with git stash and wrong branches... 2018-04-06 06:19:49 +02:00
Fabian Schlenz ebff71b208 WIP: Even moar rewriting. 2018-03-20 06:44:36 +01:00
Fabian Schlenz eea08a5559 WIP: Lots and lots of rewriting. 2018-03-16 06:59:18 +01:00
Fabian Schlenz 78031b0ff2 Don't download media files in daemon mode if download_media is false. 2018-03-15 20:49:37 +01:00
Fabian Schlenz f4d563226c Added limit and offset parameters to Database#getMessagesWithMedia in order to prevent high memory usage in downloadMedia. 2018-03-15 20:49:15 +01:00
Fabian Schlenz 968ee831f0 Added max_file_age to download only newer files. 2018-03-15 06:48:37 +01:00
Fabian Schlenz 2d409352bc WIP: Some more rewriting. 2018-03-14 06:09:24 +01:00
Fabian Schlenz 97cf26b46d WIP: Started rewriting the code to make it more null-safe. 2018-03-14 06:08:07 +01:00
Fabian Schlenz 6276651b84 Fixed anonymization of database backup debug messages. Fixes #98. 2018-03-13 06:46:59 +01:00
Fabian Schlenz f8984b25b1 Renamed IniSettings#get to IniSettings#getString. 2018-03-13 06:44:33 +01:00
Fabian Schlenz 2295ced528 Fixed a bug in IniSettings that prevented new accounts from being created. 2018-03-13 06:42:39 +01:00
Fabian Schlenz aec609e6c4 Merge branch 'feature-settings' 2018-03-13 06:32:06 +01:00
Fabian Schlenz dd99612bed Added command line switch `--list-channels` to list channels and supergroups with their ID to add them to black- and whitelists in config.ini 2018-03-13 06:31:38 +01:00
Fabian Schlenz 25a01fae4b You can now restrict the downloading of channels and supergroups by defining black- and whitelists in config.ini 2018-03-13 06:30:39 +01:00
Fabian Schlenz 077cbcebca deploy_beta.sh: Make clear that the changelog is since the last real release. 2018-03-13 06:10:44 +01:00
Fabian Schlenz a8149dfce9 deploy_beta.sh: Fixed a line causing the script to wait for STDIN (don't really understand, why) and modified the "this is just for testing, backup your data" warning. 2018-03-13 06:09:52 +01:00
Fabian Schlenz ecb225ef60 Extended .gitignore. 2018-03-12 22:02:43 +01:00
Fabian Schlenz c79336618c deploy_beta.sh: A small script to deploy the current beta version to the telegram group. 2018-03-12 22:02:14 +01:00
Fabian Schlenz e75aa2101e Fixed lots of unclosed ResultSets in Database and DatabaseUpdates. 2018-03-12 22:01:47 +01:00
Fabian Schlenz 19973818f8 Moved more settings to IniSettings. 2018-03-10 23:26:03 +01:00
Fabian Schlenz d796cb1bf0 deploy.sh: Telegram message is now HTML formatted. 2018-03-08 23:11:48 +01:00
Fabian Schlenz b0fa297a61 deploy.sh: Moved the gradle build task to earlier in the process to better be able to catch errors. 2018-03-08 23:11:14 +01:00
Fabian Schlenz a8944125b6 deploy.sh: Check out master after running the tool. 2018-03-08 22:32:51 +01:00
Fabian Schlenz d66834c3d5 deploy.sh: Use the html_url of the new release. 2018-03-08 22:32:18 +01:00
Fabian Schlenz c99766a71e deploy.sh: Create relases as published, not as draft. 2018-03-08 22:31:27 +01:00
Fabian Schlenz 79b68bd93d deploy.sh: Don't fail if the Dockerfile can't be committed (possibly because it already had the correct version in it). 2018-03-08 22:23:20 +01:00
Fabian Schlenz dcdc313c8b Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2018-03-08 22:21:39 +01:00
Fabian Schlenz ac85f06e3e Bumping the version to 1.1.3 2018-03-08 22:21:11 +01:00
Fabian Schlenz 65ae4f4a86 Fixed typo in deploy.sh 2018-03-08 22:21:07 +01:00
Fabian Schlenz 9f9d9fd183 Fixed a source for non-closed PreparedStatements. 2018-03-08 22:17:01 +01:00
Fabian Schlenz c29cd2a8ee A DatabaseUpdate can now contain a List of create_query-Strings that are used if the database is to be created from scratch. 2018-03-08 22:15:49 +01:00
Fabian Schlenz 68e5c9be2d If a new version is available, the message will be displayed a bit more prominent and with a 5 second delay. 2018-03-08 22:13:30 +01:00
Fabian Schlenz 9a9e4284d9 Modified deploy.sh to set the tag on master, not on stable, so that `git describe` for versions built from master gives usable output. 2018-03-08 22:12:29 +01:00
Fabian Schlenz 3c5a8e9d38 Deployment can not happen automatically via deploy.sh 2018-03-08 06:56:41 +01:00
Fabian Schlenz 6c6725e711
Merge pull request #51 from orschiro/patch-1
Corrects the path given in README.md.
2018-03-07 18:07:32 +01:00
Fabian Schlenz 834aaf0292 Extended .gitignore 2018-03-07 18:05:02 +01:00
Fabian Schlenz 365e38970d Modified the Dockerfile to make a bit smaller images. 2018-03-07 18:04:25 +01:00
Fabian Schlenz e40096fc44 Removed the deploy command from .travis.yml 2018-03-07 18:03:56 +01:00
Fabian Schlenz b65e624876 Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2018-03-07 18:02:58 +01:00
Fabian Schlenz 85c525ab1c Newsly download GMaps images have a red marker in them to pinpoint the exact location. 2018-03-07 17:15:59 +01:00
Fabian Schlenz da2a7d88b6 Geo Images are now shown in the HTML export. Fixes #95. 2018-03-07 17:15:20 +01:00
Fabian Schlenz b959c35bea Usind `--login --account <x>` will now directly login you to the specified account. Fixes #78. 2018-03-07 06:15:16 +01:00
Fabian Schlenz 89cca39409 IniSettings will be initialized at startup. 2018-03-07 06:12:37 +01:00
Fabian Schlenz 7fa89ab1b1 IniSettings class is now finished and also already used for GMAPS_KEY. 2018-03-07 06:09:51 +01:00
Fabian Schlenz 53fcd36e66 config.sample.ini is copied into the data directory on every run. 2018-03-07 09:48:23 +01:00
Fabian Schlenz 897435adf9
Merge pull request #77 from jakevossen5/master
Typo: Capitalized Month in README.md
2018-03-06 20:59:27 +01:00
Fabian Schlenz 74dbc9d412 Commit 0e2eeab accidentally removed the line in `--help` mentioning `--daemon`. Fixed that. 2018-03-06 06:30:50 +01:00
Fabian Schlenz 7b49153c93 Now catching (and printing) ALL exceptions in CommandLineController. 2018-03-06 06:28:37 +01:00
Fabian Schlenz 07572c0618 Extended IniSettings to be able to parse ini files. Yay. But: Still WIP! 2018-03-06 06:27:31 +01:00
Fabian Schlenz e3aaa58256 Small changes to DB_Update_9: Progress report, ORDER for the query, closing ResultSets. 2018-03-06 06:20:02 +01:00
Fabian Schlenz 6592e15b09 Bumped the version to 1.1.2 2018-03-05 18:28:54 +01:00
Fabian Schlenz bc63cfaea1 Typo in the GMaps key. Aaarg. 2018-03-05 18:28:04 +01:00
Fabian Schlenz 9678aaaee8 Started IniSettings. (WIP!) 2018-03-05 18:26:53 +01:00
Fabian Schlenz cee9fad0dd Travis: Build only master. 2018-03-05 18:04:46 +01:00
Fabian Schlenz 4e74b4c30b Renamed Settings.kt to DbSettings.kt 2018-03-05 06:56:38 +01:00
Fabian Schlenz 75786e39b4 Modified Settings class to only represent internal Settings. Also, renamed it to DbSettings. 2018-03-05 06:55:24 +01:00
Fabian Schlenz dae603a95e Removed building on JDK 6 and 8 from .travis.yml. Mainly because it didn't work. ;-) 2018-03-05 06:40:51 +01:00
Fabian Schlenz eb1f731b9d DB_Update_9 now uses chunks of messages to (massively) reduce the amount of heap space needed. 2018-03-05 06:39:30 +01:00
Fabian Schlenz f5431e805e Don't start the daemon mode if an exception happened earlier. 2018-03-05 06:36:47 +01:00
Fabian Schlenz 82d6255144 Travis now tests OpenJDK7 as well as OracleJDK8 and 9. 2018-03-03 22:50:49 +01:00
Fabian Schlenz 645d8cc315 Bumped the version to 1.1.1 2018-03-03 22:22:38 +01:00
Fabian Schlenz 14324dfa4c Merged PR #87 from @leijurv. Thanks for that. Closes #84. 2018-03-03 22:14:27 +01:00
Fabian Schlenz d4a6141862 Merge branch 'master' of https://github.com/leijurv/telegram_backup into pull-leijurv-master 2018-03-03 22:06:36 +01:00
Fabian Schlenz 12cc65aa18 Bumped mustache to v0.9.5 in order to become compatible to Java 9 and newer. Fixes #93. 2018-03-03 22:04:48 +01:00
Fabian Schlenz 8ac5744405 Modified .travis.yml to upload artifacts to gh-pages. 2018-03-03 21:24:37 +01:00
Fabian Schlenz 2324c6e0c4 Added .travis.yml 2018-03-03 21:06:41 +01:00
Fabian Schlenz 3920425d37 Added VACUUMing the database after running the migration in DB_Update_9. 2018-02-22 07:58:20 +01:00
Fabian Schlenz 42dc500514 Added Settings. 2018-02-22 07:55:53 +01:00
Fabian Schlenz de354f548d Added a database update to fix the supergroup messages that were wrongly identified as being from a channel. 2018-02-21 06:31:39 +01:00
Fabian Schlenz bb48a2f601 Using another, "better hidden" GMaps API key. TODO: Keep such keys local, making forking and developing this app harder for everyone. :-/ 2018-02-21 06:30:14 +01:00
Fabian Schlenz 3a77e91bd9 Mark files that failed to download with an 'x' in the progress window. Also modified the legend to emphasize the fact that the file will again be tried to download during the next run. 2018-02-21 06:27:35 +01:00
Leijurv 3ee90ac6c6
Reorder clause to prevent null pointer exception when response is null and try_again is true 2018-02-14 10:47:43 -08:00
Leijurv a34ba8a84c
Continue to next iteration on FLOOD_WAIT error 2018-02-14 10:46:03 -08:00
Leijurv 3911af1f88
Replace FLOOD_WAIT check with the version used in download messages 2018-02-14 10:32:54 -08:00
Fabian Schlenz 0e2eeab5b9 Added pagination for the output. 2018-01-30 18:13:34 +01:00
Fabian Schlenz ab16c44de5 Added Any.toPrettyJson(). 2018-01-29 13:19:08 +01:00
Jake Vossen 52d90ffc43
Capitalized Month 2017-12-27 20:39:31 -07:00
Fabian Schlenz 004b650762 Delegated downloading of external files to Fuel. This hopefully fixes #71. 2017-12-17 13:56:28 +01:00
Fabian Schlenz 8f1a9d391c HTMLExporter now also shows the link to the export. 2017-12-17 11:39:57 +01:00
Fabian Schlenz 93ad47639f Added a convenience method Any.toJson() 2017-12-17 11:39:31 +01:00
Fabian Schlenz 28931830eb Extended --stats to also show the different source_types from the database. 2017-12-17 11:39:07 +01:00
Fabian Schlenz c83de2ed00 Fixed the detection of supergroups and channels. Fixes #69. 2017-12-16 23:21:06 +01:00
Fabian Schlenz ab6a30c48e Rewrote Utils.anonymize(String) to String.anonymize(). 2017-12-13 19:39:27 +01:00
Fabian Schlenz 9bee01698f HTML export now shows some progress information. 2017-12-13 19:03:32 +01:00
Fabian Schlenz 6a7560cb98 Added skrink.proguard to shrink the fat jar a little bit (it goes from 14MB to 7MB). Use it by calling `proguard @shrink.proguard`. 2017-12-13 06:13:17 +01:00
Fabian Schlenz e1ad5c4e86 Merge branch 'feature-kotlin' 2017-12-13 06:11:52 +01:00
Fabian Schlenz bb180f95be Kotlin: Reformatted code. 2017-12-12 22:04:20 +01:00
Fabian Schlenz 7067f98943 Kotlin: Fixed the bugs introduced by ResultSet#getString() being able to return NULL without Kotlin knowing about that. 2017-12-12 06:09:55 +01:00
Fabian Schlenz 9bc2ed7666 Kotlin: It now compiles without any errors and / or warnings. But running it fails because raw JDBC isn't very Kotlin compatible, it seems. 2017-12-11 18:39:30 +01:00
Fabian Schlenz f75a90936d Kotlin-Code is now compilable. Now let's go on to fix all those warning... 2017-12-11 12:14:25 +01:00
Fabian Schlenz 9832d429b0 More Kotlinification. 2017-12-11 08:41:59 +01:00
Fabian Schlenz 191b2fe05a Merge branch 'master' into feature-kotlin 2017-12-11 08:29:28 +01:00
Fabian Schlenz dd0986c449 README.md now contains a link to the Telegram group. 2017-12-11 06:09:06 +01:00
Fabian Schlenz 2321fa11ee Version 1.1.0: Updated the dockerfile. 2017-12-11 06:07:16 +01:00
Fabian Schlenz 07bb29ea1c README.md: Further extensions. 2017-12-10 19:58:33 +01:00
Fabian Schlenz 37a5716e50 README.md: Explain EmptyMessages. 2017-12-10 19:53:51 +01:00
Fabian Schlenz f847c372d1 Fix: Stats are now properly displayed in Firefox. 2017-12-10 12:09:43 +01:00
Fabian Schlenz 992c456af5 Some more improvements, while I'm learning more and more about Kotlin. ;-) 2017-12-10 11:48:16 +01:00
Fabian Schlenz 584069a130 Finished AbstractFileManager, I think. 2017-12-08 08:51:44 +01:00
Fabian Schlenz 3a4508012a Started partial rewrite because the converter isn't thaaat intelligent. 2017-12-08 06:09:16 +01:00
Fabian Schlenz deb5e316cb Kotlin code is now compileable. 2017-12-07 06:12:51 +01:00
Fabian Schlenz 3309ae1338 Complete conversion. 2017-12-07 08:00:18 +01:00
Fabian Schlenz 166df82ac1 First tries. 2017-12-06 11:14:11 +01:00
Fabian Schlenz 9693371a3d Changed the order of code in CommandLineController to allow login when no database is present yet. Duh. 2017-12-06 06:09:45 +01:00
Fabian Schlenz 5328df65b3 Merge branch 'feature-channels-and-supergroups'. Closes #21. 2017-12-05 19:10:18 +01:00
Fabian Schlenz fff3483e63 Fixed stats in the HTML export for channels. 2017-12-02 15:05:04 +01:00
Fabian Schlenz 99574c6e6e Split --with-channels-and-supergroups into seperate switches --with-channels and --with-supergroups. 2017-12-01 21:17:13 +01:00
Fabian Schlenz b8d9e2a1b6 Some improvements in the output. 2017-12-01 14:13:58 +01:00
Fabian Schlenz 459de54ae7 Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2017-11-30 13:35:09 +01:00
Fabian Schlenz 7bf22f4692 Implemented backing up of supergroups and channels added. Finally. Yaaaay. Be aware: There will possibly be a lot of bugs! 2017-11-30 13:33:36 +01:00
Robert Orzanna 7dd0c044cd Use hidden folder on Linux 2017-07-09 10:22:08 +02:00
Fabian Schlenz 88eb7a412c Merge pull request #43 from Hypfer/master
Fix #41.
2017-03-23 13:10:34 +01:00
Hypfer 0de09c45dc Fix #41
I don't know why that one file is broken and nor do I care.
This could use some debug logging. Well. Whatever.
2017-03-20 20:26:40 +01:00
Fabian Schlenz 14d80a70fe '--stats' now also counts chats and users in the DB. 2017-02-26 13:20:36 +01:00
Fabian Schlenz be56f21a0f Cleanup 2017-02-23 06:52:31 +01:00
Fabian Schlenz 1a72ab8735 Expanded `--stats` to contain information about api_levels of messages in the db. 2017-02-23 06:38:43 +01:00
Fabian Schlenz 6b44a6c569 Logging now with millisecond precision. 2017-02-23 06:22:36 +01:00
Fabian Schlenz c945f05c3d Added '--stats'. 2017-02-22 18:37:28 +01:00
Fabian Schlenz 6d772a3be1 UserManager is now a Singleton as well. 2017-02-22 06:57:58 +01:00
Fabian Schlenz 42112d7607 Updated the year in the version string. 2017-02-22 06:54:01 +01:00
Fabian Schlenz a2224b326d Database is now a singleton. 2017-02-22 06:53:51 +01:00
Fabian Schlenz c5921304ed .gitignoring now dev. 2017-02-22 11:13:02 +01:00
Fabian Schlenz 92fb628b92 Some refactoring to split long methods into smaller methods. 2017-02-22 06:43:41 +01:00
Fabian Schlenz 1d8724ecb7 Catching TimeoutExceptions during mediaDownload() should now skip the file. 2017-02-21 13:41:55 +01:00
Fabian Schlenz 50ba11a86c Gradle now outputs the chosen version string. 2017-02-18 13:05:24 +01:00
Fabian Schlenz 6738e20fe4 Added timeout detection to media downloads as well. 2017-02-18 12:56:56 +01:00
Fabian Schlenz e0668926c9 More exception outputting. 2017-02-17 06:34:20 +01:00
Fabian Schlenz d9667211a4 Modified the Dockerfile to allow usage of custom parameters and arguments and to stop using `--help` all the time. 2017-02-17 06:32:52 +01:00
Fabian Schlenz a489036c2f Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2017-02-15 08:41:10 +01:00
Fabian Schlenz 2641b5ce7a Bumped the version to 1.0.6 2017-02-15 08:40:29 +01:00
Fabian Schlenz f87aef3599 Added DEPLOY.md 2017-02-15 08:39:46 +01:00
Fabian Schlenz 715b86b09b Tried skipping files in case of a timeout instead of just retrying. Then noticed that the timeout couldn't actually be thrown. Added logging for that case to investigate. 2017-02-15 08:27:11 +01:00
Fabian Schlenz 0b6a771e19 Bumped the version of sqlite-jdbc from 3.8.11.2 to 3.16.1. #32. 2017-02-14 22:19:47 +01:00
Fabian Schlenz a8325735a3 Fixed NPE from DocumentFileManager.isSticker. #30. 2017-02-14 22:10:24 +01:00
Fabian Schlenz 114a20a71c Merge branch 'master' of https://github.com/fabianonline/telegram_backup 2017-02-14 22:07:31 +01:00
Fabian Schlenz 23c6554ccb Merge pull request #37 from luckydonald-forks/master
Added simple docker file to run release in a container
2017-02-14 22:06:11 +01:00
Fabian Schlenz bc84ff1fc1 Updated README.md to mention the current rate limits. 2017-02-14 06:49:32 +01:00
Fabian Schlenz fb2db5b639 Revert e9df4eb, experimental randomization of number of messages to download, because it didn't change anything (except for slowing the download even more). 2017-02-14 06:46:40 +01:00
Fabian Schlenz e9df4ebdb4 Experiment: Randomize the number of requested messages. 2017-02-14 06:45:38 +01:00
Fabian Schlenz ec0c05fdb8 Added gradle wrappers. 2017-02-14 06:43:07 +01:00
Luckydonald 2ac8c753bb Create docker-compose.yml 2017-01-09 16:32:46 +01:00
Luckydonald 4e75d39a67 Create Dockerfile 2017-01-09 16:30:51 +01:00
271 changed files with 5062 additions and 4296 deletions

5
.gitignore vendored
View File

@ -9,3 +9,8 @@ data/
src/main/main.iml
cache4.*
src/test/test.iml
dev/
todo
deploy.secret.sh
release_notes.txt
out

13
.travis.yml Normal file
View File

@ -0,0 +1,13 @@
language: java
branches:
only:
- master
before_cache:
- rm -f $HOME/.gradle/caches/modules-2/modules-2.lock
- rm -fr $HOME/.gradle/caches/*/plugin-resolution/
cache:
directories:
- $HOME/.gradle/caches/
- $HOME/.gradle/wrapper/

12
DEPLOY.md Normal file
View File

@ -0,0 +1,12 @@
# Deploying a new version
* Update the version in the Dockerfile to the coming version.
* Commit the new Dockerfile.
* Merge into stable: `git checkout stable && git merge --no-ff master`
* Create a new tag for the new version: `git tag -a <version>`.
* Push everything to github: `git push --all && git push --tags`.
* Build it: `gradle build`.
* Create a new release on github for this version. 'Release title' has to be just the version string. Attach the newly built `telegram-backup.jar`.
* Build the docker image: `docker build -t fabianonline/telegram_backup:<version> -t fabianonline/telegram_backup:latest - < DOCKERFILE`.
* Push the docker image: `docker push fabianonline/telegram_backup`.
* Post a message into the telegram_backup users group.

15
Dockerfile Normal file
View File

@ -0,0 +1,15 @@
FROM openjdk:8
ENV JAR_VERSION 1.1.3
ENV JAR_DOWNLOAD_URL https://github.com/fabianonline/telegram_backup/releases/download/${JAR_VERSION}/telegram_backup.jar
RUN apt-get update -y && \
apt-get install --no-install-recommends -y curl && \
curl -L "https://github.com/Yelp/dumb-init/releases/download/v1.1.3/dumb-init_1.1.3_amd64" -o /bin/dumb-init && \
curl -L $JAR_DOWNLOAD_URL -o telegram_backup.jar && mkdir /data/ && \
chmod +x /bin/dumb-init && \
apt-get remove -y curl && \
apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/*
ENTRYPOINT ["/bin/dumb-init", "--", "java", "-jar", "telegram_backup.jar", "--target", "/data/"]

View File

@ -2,6 +2,16 @@
Copyright 2016 Fabian Schlenz
Licensed under GPLv3
## State of this project
The tool is working, but not really as intended: Media files in most cases can't be downloaded, message downloads are hit with 30 second delays after every 200 messages. Some users reported getting banned by Telegram without reason after using this tool (with many users not getting banned at the same time, so this could theoretically just be a coincidence).
At the same time, the official Telegram client has an official way to download one's data, which is a) officially supported and b) much, much, much faster than this tool.
Fixing this tool to at least get it to work again as planned would require more or less a complete rewrite of this code. Since I'm quite happy with the possibilities given by the official clients and don't have enough free time to spare to continue developing this project, I've decided to officially archive this tool. This is not an easy step for me, because this was my most used project and quite a lot of people wrote me nice messages and thanked me. But just keeping the user's hopes up for an update without really being able to do something doesn't seem fair. So...
So long, and thanks for all the fish. ;-) \
Fabian
## Description
This is a small Java app that allows you to download all your history from
Telegram's servers and keep a local copy of them.
@ -18,16 +28,13 @@ You can find the whole app packed into one fat jar file under
only.
* Incremental backups - if you run the tool at a later time, it will only
download new messages / media.
* You will be able to use an HTML exporter to create static HTML files
containing your chats. This feature is still in the works.
## Limitations
This tool relies on Telegram's API. Apparently they don't like people who
download lots of media files, so this app gets blocked after about 2000
downloaded media files. This is nothing bad; the app detects this and waits
the necessary amount of time as dictated by Telegram. Since this delay can
be quite large (I've seen 50 minutes), downloading lots and lots of media
can take some time.
This tool relies on Telegram's API. They started rate limiting the calls
made by this tool some time ago. As of February 2017, downloading messages
is limited to 400 messages every 30 seconds, resulting in 48,000 messages
per hour. Media download is not throttled right now, so it should be a lot
quicker.
But since this tool is designed to be able to continue it's work at any
time, you can just abort the download and continue it later - that way,
@ -44,6 +51,15 @@ Basically, you have to call it with `--login` first to login to your telegram ac
call it again with `--account <phone>` to use this account and download all
it's history. If you have just one account, you can omit this parameter.
Use `--with-supergroups` and / or `--with-channels` to also download all
messages from the supergroups / channels you have joined that have been
active in the last time.
After making a backup, call it again with `--export html` to create a few
more-or-less nice to look at HTML files containing all your chats. They will
be created in the subfolder `files` of your backup. You can just open the
file index.html in your browser to look at the files.
## Donations
I've put quite some time into this tool. If you want to donate a small
amount, you can send it via Bitcoin to *1CofYzS88iEngxMu4NqQeohWDBUHv9CNDJ* or via PayPal to
@ -55,6 +71,11 @@ link](https://www.amazon.de/ref=as_li_ss_tl?ie=UTF8&linkCode=ll2&tag=telegrambac
pay any more, but I will get a few percent of your purchase's worth from
amazon.
## Contact
If you have questions or comments or need help, you can join the
[telegram_backup Development group](https://t.me/joinchat/CXFirQenTSeGWhxnikd8tg)
at Telegram.
## Frequently asked questions
### Why do I see error messages?
The library I'm using to access Telegram has some small bugs. One of those
@ -74,7 +95,7 @@ this will be detected at the next run of this program and then tried again.
The files are being saved in your User directory in a folder named
`telegram_backup`. Under windows, this would typically be under
`C:\Users\<username>\telegram_backup`. Linux users should look unter
`/home/<username>/telegram_backup`.
`/home/<username>/.telegram_backup`.
You can change this directory by supplying `--target <dir>` when calling
Telegram_Backup.
@ -90,6 +111,15 @@ containing all your messages and other data. The folder `files` contains all
media files, named after the ID of the message they belong to. Last but not
least the folder `export` contains exported data.
### What are EmptyMessages? Why are there so many messages?
If you are a member of a normal group (non-supergroup), all messages sent to
that group are being copied to your personal messages at Telegram's servers.
If you later leave this group, those messages are being deleted at Telegram,
but since all messages are continuously numbered, you can't simply delete
them because that would leave a hole in your message numbers. So these
messages are instead replaced by EmptyMessages - those things contain zero
information, they are just saying "here was a message but it was deleted".
## Attribution
This tool uses libraries from other developers which are covered by other licenses,

View File

@ -1,24 +1,46 @@
apply plugin: 'java'
apply plugin: 'application'
mainClassName= 'de.fabianonline.telegram_backup.CommandLineRunner'
mainClassName= 'de.fabianonline.telegram_backup.CommandLineRunnerKt'
repositories {
mavenCentral()
maven {
url "https://jitpack.io"
}
jcenter()
}
buildscript {
ext.kotlin_version = '1.2.0'
repositories {
mavenCentral()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
}
apply plugin: 'kotlin'
compileKotlin {
kotlinOptions.apiVersion = "1.0"
}
dependencies {
compile('com.github.badoualy:kotlogram:666a81ef9d6707f117a3fecc2d21c91d51c7d075') {
exclude module: 'slf4j-simple'
}
compile 'org.xerial:sqlite-jdbc:3.8.11.2'
compile 'com.github.spullara.mustache.java:compiler:0.8.18'
compile 'org.xerial:sqlite-jdbc:3.16.1'
compile 'com.github.spullara.mustache.java:compiler:0.9.5'
compile 'org.slf4j:slf4j-api:1.7.21'
compile 'ch.qos.logback:logback-classic:1.1.7'
compile 'com.google.code.gson:gson:2.5'
compile 'com.google.code.gson:gson:2.8.0'
compile 'com.github.salomonbrys.kotson:kotson:2.5.0'
compile 'com.github.kittinunf.fuel:fuel:1.12.0'
testCompile 'junit:junit:4.12'
}
@ -33,7 +55,9 @@ def getVersionName= { ->
commandLine 'git', 'describe', '--tags', '--dirty'
standardOutput = stdout
}
return stdout.toString().trim()
def version = stdout.toString().trim()
logger.lifecycle("Version $version")
return version
}
ant.propertyfile(file: "$project.rootDir/build/resources/main/build.properties") {

Some files were not shown because too many files have changed in this diff Show More