BlogBooksMaker Google Apps Script to Create Blogger Blog Book (or Book parts): Description and Stable Version Info

Last updated on 9 Sep. 2023 

9 Sep. 2023 Update: I rarely use the software covered in this post now. To see the current software for creating Blogger blogbooks that I use, please visit: Short User Guide to creating Blogger Blogbooks from Backup/Export File using ExportFileFilterAndGenBook and another VBA projects' macros/code (free and open source), https://ravisiyermisc.blogspot.com/2023/09/short-user-guide-to-creating-blogger.html .

end-Update 9 Sep. 2023

BlogBooksMaker project has code (functions) that can be run from Google Apps Script Editor to create blog books (or booklets/blog book parts) of Blogger blogs. These generated blog books can have all or selected (say, by year) blog posts but not blog pages. However, blog pages content can be manually added to the generated blog books. Further, table of contents and page numbers can also manually be added to them.

The current stable version of the BlogBooksMaker project is version 12. It consists of two code files:

1) v12Code.gs.txt: https://drive.google.com/file/d/1R_gDaYuIbXl0CsGWCiAEBNmIqk8G3RVt/view?usp=drive_link .

2) v12Run-Driver.gs.txt: https://drive.google.com/file/d/1RT_2ebz5Ylg9THgiHvoKiZC8dyQqhlKw/view?usp=drive_link .

Note that a copy of the above code is provided towards the bottom of this post. Further note that all this code is free for others to use and modify.

Drive API service has also got to be added/enabled for the BlogBooksMaker project to run successfully. When running the program for the first time, a lot of permissions are requested which have to be granted. Also the code is not verified by Google and hence it is deemed unsafe. To get around this issue, in the warning dialog about "Google hasn't verified this app", the Advanced button has to be clicked and then a button to run the unsafe code (button label: Go to app-name (unsafe)) has to be clicked. Note that the message in this dialog about "Continue only if you understand the risks and trust the developer (dev-google-ac-id)" will have Google account id of the person running this code from Google Apps Script Editor as dev-google-ac-id.

Short background 

Firstly, I would like to thank the author of "bloggerToEbook" project shared under MIT license here: https://github.com/hn-88/bloggerToEbook . A related blog post of the author:  https://hnsws.blogspot.com/2022/03/google-apps-script-to-turn-blog-posts_6.html

I explored how I could alter the above project code to suit my needs better. In the course of that, I created two test projects and various versions within each to try out my alterations and that work is detailed in this post: Google Apps Script to Create Blogger Blog Books, Test Versions and Stable Version, https://ravisiyermisc.blogspot.com/2023/06/google-apps-script-to-create-blogger.html . 

This post uses some small parts of the contents of the above post. Readers who do not know much about Google Apps Script or who want to know about the background of the BlogBooksMaker project should read the above detailed post.

As mentioned above current stable version of the BlogBooksMaker project is version 12 (code links given in beginning of post).

Some background about Google Apps Script is necessary - here's the Overview page:  https://developers.google.com/apps-script/overview . The above mentioned author also has a blog post providing  links for getting started on Google Apps Script: https://hnsws.blogspot.com/2021/05/getting-started-with-google-apps-script.html .

What does BlogBooksMaker project do?

BlogBooksMaker project has code (functions) that can be run from Google Apps Script Editor to create  a blog book or blog book parts of my Blogger blogs. Note that these blog books have only blog posts and not blog pages.

Some of the functions create blog books for the whole blog. Some other functions create blog books for a single year or for default number of posts from latest post of blog.

How can users/readers use BlogBooksMaker project code for other Blogger blogs

Readers/users can copy-paste the stable version of BlogBooksMaker project having two code files (Code.gs and Run-driver.gs) into their own new Google Apps Script project. Code.gs file has the core functionality of the program which is invoked via top-level functions in Run-driver.gs file.

Readers can do minor modifications of the Run driver code file (e.g. v12Run-Driver.gs) to make these functions print the Blogger blog books the readers/users want, instead of my Blogger blogs. Note that they will have to add Drive API service to their Google Apps Script project as otherwise the program execution will fail. https://developers.google.com/apps-script/guides/services/advanced provides details of how to enable/add such services.

To run a top-level function in the program, one has to open Run-Driver.gs file in Script Editor and then choose that function as the "Run" or "Debug" function, and then click on the "Run" or "Debug" button.  Note that the program creates the blog book(s) as Google Docs document(s) in top-level folder of the user's Google Drive (the user has to have a Google account, I think).

For the first run, a lot of permissions issues crop up all of which have to be granted. Also, there is a warning about the app being not verified by Google and so unsafe! To get around this issue, in the warning dialog about "Google hasn't verified this app", the Advanced button has to be clicked and then a button to run the unsafe code (button label: Go to app-name (unsafe)) has to be clicked. Note that the message in this dialog about "Continue only if you understand the risks and trust the developer (dev-google-ac-id)" will have Google account id of the person running this code from Google Apps Script Editor as dev-google-ac-id. As I had read the code and understood quite well (though not fully, earlier on) I did not feel I was taking any big risks by granting the permissions and running the "unsafe" code/app. But I do think that users who are not familiar with Google Apps Script and have not read the code for this app., may be scared to grant all the permissions it asks for and then run the "unsafe" app. But if they don't grant these permissions then the program cannot be run successfully.

The created blog books are Google Docs created in top-level folder of user's Google Drive. These docs can be saved to PC in various formats including .docx (Microsoft Word document format). Table of Contents can be added to the document using Insert Table of Contents feature of Google Docs or Microsoft Word. Page numbers can also be added to the document.

I explored using DocumentApp.getUi().prompt() function to see if it can be used to introduce a dialog window in which users can provide parameters like blogurl, year, maxtotalposts etc. It did not work as it needs a Document context. Perhaps one can use a dummy document and in its context use this function. But I did not try that out.

Trying to have a web app implementation & deployment of this app taking parameters through Query String ran into some issues which are covered in a later section: "Why not deploy BlogBooksMaker as web app and use Query String parameters for different runs?".

About current stable version of BlogBooksMaker (version 12)

The run-functions in Run-Driver.gs (version 12) which can be invoked or run from Script Editor to create blog book(s) use the main function makeBlogBooks() in Code.gs file (makeBlogBooks() is referred to in short as MBB). These run-functions in Run-Driver.gs are:

  • function makeBlogBooksForOneYear() - Invokes makeBlogBooks() function for a particular blog for a particular year with default value of posts per blog book part (may be 50) and max of 400 posts for that year.
  • function invokeMBBWithDefaultValues() - Invokes makeBlogBooks() function with default values which will print the default blog but limit number of posts printed to default max total posts which may be 200). This function can be run from Script Editor to test MBB function.
  • function makeMySpiritualBlogBooksYearWise() - Invokes makeBlogBooks() function for my Spiritual blog - https://ravisiyer.blogspot.com/ -with year parameter in a loop. In past runs, the function has timed out due to execution timing out (crossing some execution time limit I guess) and so had to have an additional run for the remaining part of the blog (comments in function code have details of that) with first run code commented out.
  • function makeMyWorldlyBBYrlyRun1() and function makeMyWorldlyBBYrlyRun2() - Invokes makeBlogBooks() function for my Worldly blog - https://ravisiyermisc.blogspot.com/ - with year parameter in a loop. Split into two functions which will be invoked in two different runs. This may solve any execution time limit issue. Core code for both runs is in a separate function.
  • function makeMyTNSBlogBooks() - Invokes makeBlogBooks() function for my TNS blog - https://tnarayanasasthri.blogspot.com/ - which is a small one and so all of it can go into blog book(s) in one run.
  • function makeMyEklavyaSaiBlogBooks() - Invokes makeBlogBooks() function for my EklavyaSai (Indian CS & IT Academic Reform past activism) blog -  https://eklavyasai.blogspot.com/ - which is a medium sized one and so all of it can go into blog book(s) in one run.

From test runs point of view, the key test runs are v8, v11 and v12. Therefore this section covers run information for test runs of all these versions.

Version 8 (v8) is an important version in this context as it was used to successfully create blog books for whole of my spiritual blog.  The v8 run  gave me confidence that the code seemed to be stable but needed some testing before release. 

After v8, I cleaned up of the code including removing unused code to make it easier for new persons (and for even me after some period of time) to read and understand the code, and also make small modifications to it. I split the single code file (v8Code.gs) of v8, into two files with one having the main functionality code:  v9Code.gs, and the other having the code changes for each run: v9Run-Driver.gs. To create a new invocation of the main functionality code in v9Code.gs, only v9Run-Driver.gs would have to be modified leaving the main functionality code file - v9Code.gs unchanged. This process of cleaning up the code (including v9Code.gs) and making it easier to read and modify, continued in the next version v10. After v10, no code changes have been made to Code.gs associated file - i.e. v10Code.gs, v11Code.gs and v12Code.gs have same contents. Only Run-Driver.gs part of the two files changed from v10 to v12.

Version 11 was slightly modified to add further run-functions in Run-Driver.gs which resulted in version 12. 

Program runs (from Script Editor) of all the above functions (run-functions), as part of versions 8, 11 and 12 testing, completed without error or had a timeout error for which I used a workaround, and the output files (blog books) created seem to be as expected, with only one exception. Note that all the documents produced by the runs have not been studied in detail as of now and so when they are studied (as I plan to do so in future), problems/issues could be discovered.

The exception details are related to program run of function makeMyWorldlyBBYrlyRun2() . In document or blog book, "ravisiyermisc.blogspot.com, year: 2009, part 1", pics are missing!!! This needs further investigation. But in document or blog book, "ravisiyermisc.blogspot.com, year: 2015, part 1" pics are shown! Is there an issue with pics format of older than 2015 posts which the Google Apps Script code is not able to handle properly? As of now, I don't know what the cause of the issue is. But I think the problem is not so significant for my needs. I can work around the problem by printing the individual posts with pics for year 2009 of above mentioned blog, as PDF document(s). So I have decided, as of now, not to spend more time to investigate the issue. Interested readers/users are welcome to do the investigation and share their findings.

Top-Level Shared Folder with all related Google Apps Script projects' code and data - BlogBooksScriptCode:  https://drive.google.com/drive/folders/1gS2uZOixK3yBaw67wgqKHCbvESXtsh4F?usp=drive_link . [While I have the option of sharing the project files themselves, I am not sure what the security implications are and so am not doing that. But I am sharing all the code and most of the related data (output files, run info. and execution logs).]

Given below are extract(s) related to versions 12, 11 and 8 from the aforementioned post: Google Apps Script to Create Blogger Blog Books, Test Versions and Stable Version,  https://ravisiyermisc.blogspot.com/2023/06/google-apps-script-to-create-blogger.html (extracts copy-pasted on 7 Jul. 2023).

--- start extracts from blog post (slightly edited for formatting) ---

BlogBooksMaker project

This section is in reverse chronological order.

The folder corresponding to this project is shared here:  https://drive.google.com/drive/folders/1jipFEeigwZ4W_LBtEF_jiWZXzFY41Xcs?usp=drive_link .

7th July 2023 Update start:

v12 version of BlogBooksMaker Project

Added functions to Run-Driver.gs. Did not add anything to Code.gs as far as I can recall.

v12 code files (2):

The makeMyTNSBlogBooks() function of Run-Driver.gs was run to make the full TNS blog book parts. 

20230706-Full-TNSBlog folder contains the doc file created, and run info-exec log of the above run: https://drive.google.com/drive/folders/1aWSHHEB63Smov8QvuCp_BdmAw7XuULjg?usp=drive_link 

v12FullTNSBlogRunInfo-ExecLog.txt shows run info. and execution log of makeMyTNSBlogBooks() function: https://drive.google.com/file/d/1b7cE9b_dn6pVNIQwEY3U4kkDVcMBrM7i/view?usp=drive_link .

Given below are some extracts from its initial part (slightly edited for better formatting):

Top-level Execution log:
Deployment   Function                               Type     Start Time                              Duration    Status
Head        makeMyTNSBlogBooks     Editor Jul 6, 2023, 12:18:58 AM 12.575 s   Completed
==========================
Output doc seems to be as expected.
--- end extracts ---

Then the makeMyEklavyaSaiBlogBooks() function of Run-Driver.gs was run to make the full EklavyaSai blog book parts. 

20230706-Full-EklavyaSaiBlog folder contains the doc files created, and run info-exec log of the above run:  https://drive.google.com/drive/folders/1cqDKT-QvQyu0s_cegk5aUai7u_SoC_mR?usp=drive_link

v12FullEklavyaSaiBlogRunInfo-ExecLog.txt shows run info. and execution log of makeMyEklavyaSaiBlogBooks() function:  https://drive.google.com/file/d/1ROly9sm9_IFH_mqgmmsMHPIh7CRBzgyM/view?usp=drive_link .

Given below are some extracts from its initial part (slightly edited for better formatting):

Top-level Execution log:
Deployment Function                                    Type    Start Time                          Duration  Status
Head     makeMyEklavyaSaiBlogBooks Editor Jul 6, 2023, 12:47:38 AM  43.032 s  Completed
==========================
5 docs from "eklavyasai.blogspot.com part 1" to "eklavyasai.blogspot.com part 5" were created. Output docs seems to be as expected.

--- end extracts ---

7th July 2023 Update end

=====================================================

5th July 2023 Update start:

v11 version of BlogBooksMaker Project

Added functions to Run-Driver.gs. Did not add anything to Code.gs as far as I can recall.

v11 code files (2):

v11InitialRunInfo-ExecLogs.txt file shows run info. and execution logs for test runs of two Run-Driver.gs functions of above version:  https://drive.google.com/file/d/1ftIRCE8k628AfbvPFWjNW3LbSUYcTCeQ/view?usp=drive_link

After the above successful runs, the makeMyWorldlyBBYrlyRun1() function of Run-Driver.gs was run, followed by separate run of makeMyWorldlyBBYrlyRun2() function of Run-Driver.gs to make the full Worldly blog book parts. The two separate runs were to avoid running into execution time limit issues.

20230705-Full-WorldlyBlog contains the doc files created, and run info-exec logs of the above runs: https://drive.google.com/drive/folders/1wiOVZAAQA7XovdCxl4a7DP_0t9VOM9jb?usp=drive_link

v11FullWorldlyBlogRun1Info-ExecLog.txt shows run info. and execution log of makeMyWorldlyBBYrlyRun1() function: https://drive.google.com/file/d/1clG8VJt6-6Ktpu9plEk0H7RVdIosYj8Q/view?usp=drive_link .

Given below are some extracts from its initial part (slightly edited for better formatting):

Top-level Execution log:
Deployment    Function                                  Type        Start Time                        Duration    Status
Head makeMyWorldlyBBYrlyRun1 Editor Jul 5, 2023, 7:33:34 PM 207.687 s    Completed
==========================
19 output doc files were created from "ravisiyermisc.blogspot.com, year: 2023, part 1" to "ravisiyermisc.blogspot.com, year: 2017, part 5". Sample checking of few docs from the set indicate that these output docs are as expected.

--- end extracts ---

v11FullWorldlyBlogRun2Info-ExecLog.txt shows run info. and execution log of makeMyWorldlyBBYrlyRun2() function:  https://drive.google.com/file/d/1fjrbNJXtyIyBUAlIKF2mOxGwiYBarKdw/view?usp=drive_link .

Given below are some extracts from its initial part (slightly edited for better formatting):

Top-level Execution log:
Deployment Function                                     Type         Start Time                     Duration     Status
Head makeMyWorldlyBBYrlyRun2 Editor Jul 5, 2023, 8:48:54 PM 61.588 s    Completed

==========================
10 output doc files were created from "ravisiyermisc.blogspot.com, year: 2016, part 1" to "ravisiyermisc.blogspot.com, year: 2007, part 1". Sample checking of few docs from the set indicate that these output docs are mostly as expected but with some issues.

"ravisiyermisc.blogspot.com, year: 2009, part 1" pics are missing!!! Needs further investigation.
But "ravisiyermisc.blogspot.com, year: 2015, part 1" pics are shown! Is there an issue with pics format of older than 2015 posts which the Google Apps Script code is not able to handle properly?

--- end extracts ---

...

v8 version of BlogBooksMaker Project

v8 code: https://drive.google.com/file/d/1EZBnhv2BJjTK57yfih4T6ZrPPY75A_z4/view?usp=drive_link

v7 to v8 changes: Addition of mainbooktitle parameter; addition of function makeMySpiritualBlogBooksYearWise(); removal of some commented code about default value settings; adding code to add year string as part of book title if year has been specified; doGet() function being commented out as web app was behaving strangely.

The run info. and log file (Run3-RunInfo-ExecLog.txt) is here:   https://drive.google.com/file/d/12iSxLmSAQ_nCMEzT7XDFphujLNrgU4D7/view?usp=drive_link

Given below are extracts from initial part of above file:

4 Jul 2023

Two runs:

1st run, 4 Jul 2023: Produced blog books from 2023 to 2016 - need to check them out. Execution timed out after 2015 part 1 before completion of part 2 though part 2 file also was created. The relevant execution log entry is: "Jul 4, 2023, 1:25:46 AM Error Exceeded maximum execution time". To prevent confusion I deleted 2015 parts 1 & 2 files of this run.

2nd run, 4 Jul 2023: Produced blog books from 2015 to 2013 - need to check them out. Program completed normally as blog starts from year 2013.

------------

These two runs used v8Code.gs.txt and its slightly earlier version (as calling of myFunction related code in function makeMySpiritualBlogBooksYearWise() had to change to handle timeout error in first run - that's all, no other changes).

===================================

Top-level info. in Execution Log for the 2 runs:

Deployment Function                                             Type     Start Time                       Duration  Status

Head makeMySpiritualBlogBooksYearWise Editor   Jul 4, 2023, 1:32:18 AM 83.669 s  Completed

Head makeMySpiritualBlogBooksYearWise Editor   Jul 4, 2023, 1:19:46 AM 360.164 s Timed Out

==== end top-level info ====

--- end extracts from file ---

Note that the output of this run is shared in subfolder (20230703-Full-SpiritualBlog):  https://drive.google.com/drive/folders/1R-g6ViFpFwbbNRWVR-gK3zdGinTdYG1_?usp=drive_link .

I encountered a bug/issue with downloading these multiple Google Docs as a set onto my PC. From the small file that captures the issue (DownloadFileSetBug-Issue.txt),   https://drive.google.com/file/d/17S9xF3D58cBBfdRzZXS92-fw23rEdL8O/view?usp=drive_link :

ravisiyer.blogspot.com, year_ 2015, part 2.docx was extracted from zip file drive-download-20230703T204143Z-001.zip which was made by Google Drive when I chose to download multiple files as one download. Unfortunately the .docx file does not have one image in the post: Great Photo of Sathya Sai delight & wonder at his own miraculous creation of Krishna idol; Published: 2015-10-27 (starting) on page 3. But the Google Doc on Google Drive has it! So, at times at least, there seems to be a problem in downloading multiple Google Doc files as a zip.
--- end file contents ---

================

--- end extract from blog post ---

Why not deploy BlogBooksMaker as web app and use Query String parameters for different runs?

Well, that's what I felt I should do. The initial versions of BlogBooksMaker were used as web app deployment and Query String parameters provided a way to run the code with different parameters (like blogurl, year, maxpostsperpart and maxtotalposts) to generate different blog books. It worked well with limited data I was using for testing these initial versions. v1 and v2 were used as web apps. v7 version  was also used a web app and was the big stumbling block for me, as I encountered very strange behaviour.

v6 version, if I recall correctly, was run from Script Editor to create blog book(s) for the whole of my spiritual blog. It ran well for around 1100 posts but after that it tripped up with an "Internal Error" with the run getting aborted. Its output of 23 Google docs was not checked in detail. 

v7 version was the same trial run to create blog book(s) for the whole of my spiritual blog but running the program as web app and providing appropriate Query String parameters. While it tripped up on the same point as the above v6 version run, the v7 version (web app) behaved very strangely after that with automatic reinvocations and creating a ton of files! The run raised many questions about stability of web app as against running from script editor. I decided to stop further work on web app, and consider only using script editor runs for the near future for this project.

Interested readers/users are welcome to use the web app code of v7 version, modify it and test out the modified version. If it works it will be great if they publicly share the code.

Detailed information about all the above mentioned versions are provided in the "BlogBooksMaker project" section of the post: Google Apps Script to Create Blogger Blog Books, Test Versions and Stable Version, https://ravisiyermisc.blogspot.com/2023/06/google-apps-script-to-create-blogger.html .

Current Stable Version Code Copy

As mentioned in the top of this post, the current stable version of the BlogBooksMaker project is version 12. It consists of two code files:

1) v12Code.gs.txt: https://drive.google.com/file/d/1R_gDaYuIbXl0CsGWCiAEBNmIqk8G3RVt/view?usp=drive_link .

2) v12Run-Driver.gs.txt:  https://drive.google.com/file/d/1RT_2ebz5Ylg9THgiHvoKiZC8dyQqhlKw/view?usp=drive_link .

I felt it appropriate to provide a copy of the above two files in this post itself and so have given them below.

--- Start v12Code.gs (copy-pasted on 8 Jul 2023) ---

// This code is based on the code available here: https://github.com/hn-88/bloggerToEbook which is
// licensed under "MIT License" .."Copyright (c) 2022 hn-88".
// The details of that license can be viewed here: https://github.com/hn-88/bloggerToEbook/blob/main/LICENSE
// As the license allows others "the rights to use, copy, modify, merge, publish" the software, I have used
// it in my code given below and associated web app.
// The license for this modified version (code below) (modifications done by me, Ravi S. Iyer) is provided
// in my blog post:
// All my blog data and books publicly accessible on Google Drive; Permission for free reuse,
// https://ravisiyer.blogspot.com/p/all-my-blogbooks-publicly-accessible-on.html .
// To know more about how I modified this code and tried it out, please visit my blog post:
// Google Apps Script to Create Blogger Blog Books, Test Version - In Progress Post, 
// https://ravisiyermisc.blogspot.com/2023/06/google-apps-script-to-create-blogger.html ,
// created on 28 Jun 2023.
//
/* I (Ravi S. Iyer) am now an obsolete software developer as I have stayed away from coding for around, if
  not over, a decade now, barring very tiny JavaScript tweaking of others' code to suit my needs. Prior to
  this exploration I had never used Google Apps Script - so the development platform as well as the 
  development environment is new to me. I want to limit the time I spend on this work. So I am looking at
  quick fix modifications to the code which may be bad coding style. I also may not be calling the right API 
  functions or calling them the right way - I just don't have the time to read up on the development 
  platform / API barring just quick viewing of reference pages and help I get from Google Search result 
  links, to figure out what I should try. I am not drastically changing the original code and am trying to 
  stick to its style.  
  I am focusing on modifications for my specific needs and not for any general purpose needs. So my code may 
  have bugs and problems which do not come into play when I am using it for my specific needs but come into 
  play when others use it for different needs. I am not in a position now to help fixing such issues. Other 
  developers are absolutely welcome to do such fixes or other changes and re-publish their version of this 
  software. */

var contenthtml = '';
const DEF_BLOG_URL = 'https://ravisiyermisc.blogspot.com/';
const DEF_BOOK_TITLE = 'BlogBook'; 
const DEF_MAX_POSTS_PER_PART = 50;  // Release version default value
               // Higher default value like 100 results in program run failures at times. Error message
               // usually the following or similar:
               // GoogleJsonResponseException: API call to drive.files.insert failed with error: Bad Request
const DEF_MAX_TOTAL_POSTS = 100;     // Release version default value
function makeBlogBooks(blogurlarg, yr, maxpostsperpart, maxtotalposts, fetchurlmainpart, mainbooktitle) {
  Logger.log("makeBlogBooks arguments: blogurlarg = " + blogurlarg + ", yr = " + yr + ", maxpostsperpart = "
  + maxpostsperpart + ', maxtotalposts = ' + maxtotalposts + ', fetchurlmainpart = ' + fetchurlmainpart
  + ', mainbooktitle = ' + mainbooktitle);
  var apicall;
  var blogurl;
  var maxresults;
  if (maxtotalposts == null)
    maxtotalposts = DEF_MAX_TOTAL_POSTS;
  if (mainbooktitle == null)
    mainbooktitle = DEF_BOOK_TITLE;
  if (fetchurlmainpart == null)
  {
    if (blogurlarg == null)
      blogurl = DEF_BLOG_URL;
    else
      blogurl = blogurlarg;
    if (maxpostsperpart == null)
      maxpostsperpart = DEF_MAX_POSTS_PER_PART;
    
    maxresults = +maxpostsperpart;
    if (maxtotalposts < maxpostsperpart)
      maxtotalposts = maxpostsperpart; 
    if (yr == null)
      apicall = blogurl+'feeds/posts/default'+'?max-results='+ maxresults + '&alt=json';
    else {
      var prevyr = +yr - 1;
      var nextyr = +yr + 1;
      apicall = blogurl+'feeds/posts/default'+'?max-results='+ maxresults + 
      '&published-min=' + prevyr +'-12-31T00:00:00-08:00&published-max='+ nextyr +
      '-01-01T23:59:59-08:00&alt=json'; 
      // Timezone +05:30 trips up the program. I think the + character needs to be escaped but
      // don't know how. So as a hack-fix I am adding a day before min and a day after max
    }
  }
  else {
      apicall = fetchurlmainpart;
      //maxresults has to be extracted from fetchurlmainpart if present, or set to default
      var x = parsemaxresults(fetchurlmainpart);
      if ((x == 0) || isNaN(x) || (x == null))
        maxresults = DEF_MAX_POSTS_PER_PART;
      else 
        maxresults = x;
      
      Logger.log ("After checking for max-results string in fetchurlmainpart, " +
      "maxresults variable set to: " + maxresults);
      
  }
                                 
  var options = {
    method: 'GET',
    muteHttpExceptions: true,
    
  };
  var i = 0; // index of json items in response  
  var postindex = 1; // index of total number of posts in feed, which we fetch maxpostsperpart at a time
                      // Ravi: I think it is a 1 based index used for start_index part of URL
                      // in UrlFetchApp.fetch() call
  var response;
  var loopindex = 1;
   Logger.log("apicall variable = " + apicall);
  do { 
    i = 0;
    response = UrlFetchApp.fetch(apicall+'&start-index='+postindex, options);
    
    var json = JSON.parse(response.getContentText());
    if (json.feed.entry == null) {
      Logger.log("Last UrlFetchApp.fetch() call returned 0 entries. Job is done. " + 
      "Break out of loop to avoid creating empty Google Doc output file.");
      break;
    }
    for ( i in json.feed.entry) {
      var pubdatetime = json.feed.entry[i].published.$t;
      var pubdate = pubdatetime.substring(0,10);
      for (var j = 0; j < json.feed.entry[i].link.length; j++) {
      if (json.feed.entry[i].link[j].rel == 'alternate') {
        break;
        }
      }
      var ListUrl = json.feed.entry[i].link[j].href;
      contenthtml+='<h1>'+json.feed.entry[i].title.$t
            + '; Published: '+pubdate
            +'</h1>'+'Post link (URL) on blog: <a href="' +ListUrl+'">' + ListUrl +'</a>'+'<br/><br/>'
            +json.feed.entry[i].content.$t
            +'<br/>===========================End of Post============================<br/>'
            + '<span style="break-after: always;" />'; //Ravi: Does not create page break in
                                                       // Google Docs document
      Logger.log("Post Title (80 chars), pubdate of this fetch: '" +
      json.feed.entry[i].title.$t.substring(0,80) + "', " + json.feed.entry[i].published.$t) ;
      Logger.log('%s in this fetch, %s overall',(+i+1), (+postindex+(+i)));
    
    }
    postindex = +postindex+(+i)+1; //Ravi: postindex is used for start_index which seems
                                    // to be a 1 based index.
                                    // if maxresults is 10, then for first iteration of fetch which returns
                                    // maxresults, prior to execution of this code statement,
                                    // i will be 9 (as it is a 0 based index) and postindex will be 1.
                                    // After statement execution, postindex will 1+9+1 = 11 which seems to be
                                    // the right value for start_index for next call to fetch to get the
                                    // next set of maxresults posts.
    try {
      var ablob = Utilities.newBlob(contenthtml, MimeType.HTML, "asset.html");
      Logger.log("contenthtml.length = " + contenthtml.length);
      var booktitle;
      if (yr == null)
        booktitle = mainbooktitle +' part '+ loopindex;
      else
        booktitle = mainbooktitle + ', year: ' + yr +', part '+ loopindex;
        var Dx = Drive.Files.insert(
        { title: booktitle, 
        mimeType: MimeType.GOOGLE_DOCS },
        ablob);
      var AssetGDocId = Dx.id;
      Logger.log('Wrote "%s" to GDoc.', booktitle );
    }
    catch(err){
      Logger.log('Error is %s', err);
      return ('Failure to write output Google Docs file. Error Message: ' + err);
    }
    loopindex++;
    contenthtml='';
    if (postindex > maxtotalposts)
    {
      Logger.log('postindex = ' + postindex + ' which is > maxtotalposts = ' + maxtotalposts
       + '. So time to exit loop and finish program run.');
       break;
    }
    
    Logger.log('i value just before while (i>maxresults-2) condition: ' + i +
    ', maxresults = ' + maxresults);
  //} while (i>98); //Ravi: I think it is testing the case where the last
                    // set of results have been returned. Note that earlier max-results was always 100
  } while (i>maxresults-2); //Ravi if maxresults is 4, i will be 3 when it comes to this point  
                            // in first iteration of loop if fetch gives 4 results in first iteration.
                            // But if fetch gives less than 4 results then we know we don't need to call
                            // fetch again. so the test has to be is that so long as i is 3 we can loop back.
                            // To generalize, the test should be while (i==maxresults-1).
                            // The original code uses a slightly different test which should also work of
                            // while (i>98) when max. number of posts requested in the fetch is 100
                            // To use the same style here, the condition has to be:
                            // while (i>maxresults-2) .
  return ("Seems to be successful makeBlogBooks() function invocation. See Execution Log for details.");
}

function parsemaxresults(fetchurlmainpart) {
console.log("parsemaxresults() called with arg = '" + fetchurlmainpart + "'")
var indexOfmaxresults = fetchurlmainpart.indexOf("max-results");
console.log('The index of max-results is ' + indexOfmaxresults);
var maxresultsstr = fetchurlmainpart.substring(indexOfmaxresults);
console.log("maxresultsstr = '" + maxresultsstr +"'");
var indexOfampchar = maxresultsstr.indexOf("&");
if (indexOfampchar == -1) {
  // No & character, so max-results is last parameter
  // But can there be spaces at the end? If so, trim end space part.
  var indexOfspacechar = maxresultsstr.indexOf(" ");
  if (indexOfspacechar == -1) {
    // No space char and maxresultsstr has only max-results parameter
    console.log("No space char and no further parameters, maxresultsstr = '" + maxresultsstr +"'");
  }
  else {
    maxresultsstr = maxresultsstr.substring(0,indexOfspacechar);  
    console.log("No further parameters and after truncating part from first space character, " + 
    "maxresultsstr = '" + maxresultsstr + "'");
  }
}
else {
  maxresultsstr = maxresultsstr.substring(0,indexOfampchar);
  console.log("After truncating part from first & character, maxresultsstr = '" + maxresultsstr+"'");
}
// Now to get the digits or number in max-results parameter
var indexOfeqchar = maxresultsstr.indexOf("=");
if (indexOfeqchar == -1) {
  // error in format of parameter
  console.log("No equal to = character in maxresultsstr = '" + maxresultsstr+"'");
  // abort or give default value to maxresultsnum
}
else {
  maxresultsstr = maxresultsstr.substring(indexOfeqchar+1);
  console.log("After extracting part after = character in maxresultsstr = '" + maxresultsstr+"'");
  var x = parseInt(maxresultsstr);
  if (isNaN(x))
      console.log("The specified parameter is not a number.");
  else 
      console.log("The specified parameter is a number with value = " + x);
}
return (x);
}

--- End v12Code.gs (copy-pasted on 8 Jul 2023) ---

....

--- Start v12Run-Driver.gs (copy-pasted on 8 Jul 2023) ---

// This file has various invocations of makeBlogBooks() function which can be run from Script Editor
// The parameters of makeBlogBooks() can be made out from the parameter names in function call code below
  //var funcReturnMsg = makeBlogBooks(blogurlparm, year, maxpostsperpart, maxtotalposts,
  // fetchurlmainpart, mainbooktitle);
// The license for the code in this file (code below) (authored by me, Ravi S. Iyer) is provided
// in my blog post:
// All my blog data and books publicly accessible on Google Drive; Permission for free reuse,
// https://ravisiyer.blogspot.com/p/all-my-blogbooks-publicly-accessible-on.html .
// Initially I had tried to use web app deployment and pass parameters through query string to
// the main function (now called makeBlogBooks). But I faced some unusual issues which led me to
// take the decision to drop the web app approach. However, I plan to document the code I had written
// for the web app implementation as well as the issues I faced in my blog post:
// Google Apps Script to Create Blogger Blog Books, Test Version - In Progress Post, 
// https://ravisiyermisc.blogspot.com/2023/06/google-apps-script-to-create-blogger.html ,
// created on 28 Jun 2023. You may visit that post to check that code and issues out. Readers are 
// welcome to fix the issues and make the web app deployment stable and share that freely with others.

// Invokes makeBlogBooks (MBB) for a particular blog for a particular year with default value of posts
// per blog book part (may be 50) and max of 400 posts for that year.
// I think this function will not encounter execution time limits for most blogs and so is safe to run
// To run it for a different blog and year, simply change the parameters in the single function call code
// below: 1st parameter - blog url, 2nd parameter - year, 4th parameter need not be changed unless blog 
// has more than 400 posts for that  year, 6th parameter - main part of blog book title
function makeBlogBooksForOneYear() {
    var funcReturnMsg = makeBlogBooks("https://ravisiyer.blogspot.com/", 2023, null, 400, null,
    "ravisiyer.blogspot.com"); 
  Logger.log(funcReturnMsg);
}
// Invokes makeBlogBooks (MBB) with default values which will print the default blog but limit
// limit number of posts printed to default max total posts (which may be 200)
// This function can be run from Script Editor to test MBB function.
function invokeMBBWithDefaultValues() {
  var funcReturnMsg = makeBlogBooks(null, null, null, null, null, null);
  Logger.log(funcReturnMsg);
}
// Invokes makeBlogBooks (MBB) for my Spiritual blog with year parameter in a loop.
// In past runs, the function has timed out due to execution timing out (crossing some execution time limit
// I guess) and so had to have an additional run for the remaining part of the blog (comments below have
// details of that) with first run code commented out.
function makeMySpiritualBlogBooksYearWise() {
  var year;
  var funcReturnMsg;
  /*for(year = 2023; year > 2012; year--){
    funcReturnMsg = makeBlogBooks("https://ravisiyer.blogspot.com/", year, null, 400, null,
    "ravisiyer.blogspot.com"); 
    Logger.log(funcReturnMsg);
  }*/
  // 4 Jul 2023: Above code produced blog books from 2023 to 2016 - need to check them out.
  // Execution timed out after 2015 part 1 before completion of part 2 though part 2 file also was created.
  // So code below starts from 2015 counting down. To prevent confusion I deleted 2015 parts 1 & 2 files
  // of above run.
  for(year = 2015; year > 2012; year--){
    funcReturnMsg = makeBlogBooks("https://ravisiyer.blogspot.com/", year, null, 400, null,
    "ravisiyer.blogspot.com"); 
    Logger.log(funcReturnMsg);
  }
  // 4 Jul 2023: Above code produced blog books from 2015 to 2013 - need to check them out. Program completed
  // normally as blog starts from year 2013.
}
// Invokes makeBlogBooks (MBB) for my Worldly blog with year parameter in a loop.
// Split into two functions which will be invoked in two different runs. This may solve any execution time
// limit issue. Core code for both runs is in a separate function
function makeMyWorldlyBBYrlyRun1() {
  makeMyWorldlyBlogBooksYearWise(2023, 2017);
}
// 5 Jul 2023: Above function run did not have any execution timeout and created 19 output doc files from
// "ravisiyermisc.blogspot.com, year: 2023, part 1" to // "ravisiyermisc.blogspot.com, year: 2017, part 5".
// Sample checking of few docs from the set indicate that these output docs are as expected.
function makeMyWorldlyBBYrlyRun2() {
  makeMyWorldlyBlogBooksYearWise(2016, 2007);
}
// 5 Jul 2023: Above function run did not have any execution timeout and created 10 output doc files from
// "ravisiyermisc.blogspot.com, year: 2016, part 1" to "ravisiyermisc.blogspot.com, year: 2007, part 1".
// Sample checking of few docs from the set indicate that these output docs are mostly as expected but 
// with some issues.
// "ravisiyermisc.blogspot.com, year: 2009, part 1" pics are missing!!! Needs further investigation.
// But "ravisiyermisc.blogspot.com, year: 2015, part 1" pics are shown! Is there an issue with pics format
// of older than 2015 posts which the Google Apps Script code is not able to handle properly?
function makeMyWorldlyBlogBooksYearWise(yearStart, yearEnd) {
  var year;
  var funcReturnMsg;
  for(year = yearStart; year >= yearEnd; year--){
    funcReturnMsg = makeBlogBooks("https://ravisiyermisc.blogspot.com/", year, null, 400, null,
    "ravisiyermisc.blogspot.com"); 
    Logger.log(funcReturnMsg);
  }
}
// TNS blog is a small one and so all of it can go into blog book(s) in one run
function makeMyTNSBlogBooks() {
  funcReturnMsg = makeBlogBooks("https://tnarayanasasthri.blogspot.com/", null, null, null, null,
  "tnarayanasasthri.blogspot.com"); 
  Logger.log(funcReturnMsg);
}
// 6 Jul 2023: Above function run did not have any execution timeout and created 1 output doc file - 
// "tnarayanasasthri.blogspot.com part 1". This document seems to be as expected.
// EklavyaSai (Indian CS & IT Academic Reform past activism) blog is a medium sized one and so all of it
// can go into blog book(s) in one run.
function makeMyEklavyaSaiBlogBooks() {
  funcReturnMsg = makeBlogBooks("https://eklavyasai.blogspot.com/", null, null, 400, null,
  "eklavyasai.blogspot.com"); 
  Logger.log(funcReturnMsg);
}
// 6 Jul 2023: Above function run did not have any execution timeout and created 5 output doc files from 
// "eklavyasai.blogspot.com part 1" to "eklavyasai.blogspot.com part 5".
// These documents seem to be as expected.

--- End v12Run-Driver.gs (copy-pasted on 8 Jul 2023) ---

Comments

Archive

Show more