Pages

Friday, March 13, 2015

Going going gone! Apps Script on tablets

Anyone else get stuck with fixing their familys computer problems? Recently I had a family technical issue that was actually fun to solve and didnt involve removing malware. My family owns Rasmus Auctioneers, a global auctioneer company based in Alexandria, Virginia. The company uses Google Apps for their entire business workflow, so their documents, calendars, etc. are all connected.

We manage the shipping process on-site using a tablet. In this fast-paced environment and with limited time, we need a quick, easy solution. When an auction ends, the clients fill out a Google Form that saves their shipping and payment information to a Google Spreadsheet. This information then needs to be sent to a specific mover for processing -- but the mover doesnt need all the data, just a few important pieces. All of this needs to happen with as little human intervention as possible.

Our solution was to create an Apps Script within the spreadsheet, which means our new functionality works in the tablets used on the auction floor. The function below runs when the sheet is opened, automatically sorting the sheet to put the newest client on top. A simple dialog determines which data is important for a particular auction; the script then finds the data we need, sends it to the right person, and adds a calendar event as a reminder.

function sendData() {
// Sort the rows
var sheet = SpreadsheetApp.getActiveSheet();
sheet.sort(4);

// What row do we want?
var myValue = Browser.inputBox("What row?");

// Variables
var emailRecipients = "me@mydomain.com";
var emailSubject = "Moving data for ";
var valueToShow = "";

// Get cell values
var clientName = sheet.getRange("B" + myValue).getValue();
var movingDate = sheet.getRange("D" + myValue).getValue();
valueToShow += "Name: " +
sheet.getRange("B" + myValue).getValue() + "
";
valueToShow += "Moving Date: " +
sheet.getRange("D" + myValue).getValue() + "
";
valueToShow += "Size: " +
sheet.getRange("E" + myValue).getValue() + "
";

// Send email
MailApp.sendEmail(emailRecipients, emailSubject + clientName,
valueToShow);

// Add to calendar
CalendarApp.createAllDayEvent("Move today! " + clientName,
new Date(movingDate));
}

Looking at the source code, we can see how we collect the data from the relevant columns, add them to a variable, and in one line send it via Gmail. Adding a reminder to our calendar also takes just one line.

Do you have a repetitive process that bugs you? Wish there was a way to automate something? You might be able to solve your problems the same way I helped out my family -- with Apps Script! Feel free to leave a comment on some ideas or suggestions for future projects.


Tom Fitzgerald profile | blog

Tom Fitzgerald is an engineer for Google at their Mountain View headquarters. While attempting to keep his HTML5 blog up to date he works on various web development side projects. Originally from Maryland he supports the Baltimore Ravens & Orioles!

Read more »

Thursday, March 12, 2015

Approval Workflow using Apps Script

Editor’s Note: This blog post is authored by Dito’s Steve Webster who is a Google Apps Script Top Contributor - Saurabh Gupta

Ethics Disclosure Review Workflow

Recently a company who operates retail stores throughout a few states reached out to Dito. When their associates conduct business discussions with vendors or customers where monetary exchanges are involved, their Ethics Disclosure policy requires a log for each event, a supervisor approval, and committee review.

The customer’s existing disclosure review process was cumbersome and time consuming. The employees would add a disclosure review request to a spreadsheet with 29 columns. The new review request would then be reviewed by their supervisor. The supervisor and the employee would go back and forth exchanging emails and making changes to a dense spreadsheet until an approval was granted. It was clear that the customer needed a workflow solution. They decided to hire Dito to build a workflow solution based on Google Apps Script.

Workflow Solution based on Google Apps Script

To make the process more user friendly and productive, Dito decided to build a user interface to collect ethics disclosure events, make updates, and automate the routing of email notifications. Writing a Google Apps Script to create a user interface (UI), enabled associates to interact with their contacts to select their supervisor’s email address and simplify the data collection with list boxes. The script sends approval emails with HTML form radio buttons, text box, approve/decline buttons, and a “Post” command to invoke other workflow scripts. Below are some of the main design points for this Approval Workflow script.


1. Disclosure Review Workflow

The Disclosure Review workflow requires (a) Associates to fill out the Ethics Disclosure form. (b) Supervisor to either approve or decline the submission. (c) If supervisor approves, the Ethics Disclosure Committee is notified. (d) If supervisor declines, the associate is notified to make corrections. (e) After improving the submission, the workflow repeats itself.


2. Disclosure Review Request Form

Dito developed a custom review request form. A form was developed using Google Apps Script’s UI Services. The form provides the ability to look up data to populate such things as a drop-down list box. This form allowed real-time email look-ups by using the Apps Script’s Contacts service.  First efforts included per character look-ups in a list box, but since they have over 1,000 users, it was best to enter the first and/or last name of their supervisor before initiating the look-up (see code snippet below).


var byName = ContactsApp.getContactsByName(searchKey);
for (var i in byName) {
var emailStr = byName[i].getPrimaryEmail();
// If there is no primary email, try again for the next email
if (emailStr == null)
var emailStr = byName[i].getEmails()[0].getAddress();
// If emailStr is still null, try again by getting the next email
if (emailStr == null)
var emailStr = byName[i].getEmails()[1].getAddress();
}

Another dynamic field was the “activity type”.  Depending on the selection more form fields are displayed. For example, if the activity type is “Meals”, display a list box to select lunch or dinner.


3. Approve or Reject directly in Gmail

When an associate submits his/her review request by using the custom form within a spreadsheet, their supervisor receives an email with easy-to-read HTML formatted results.  The approval decision, as well as a comment field (e.g. decline reason), is made within the email. This is more productive and prevents unnecessary back and forth into the spreadsheet.

If the request is declined by the supervisor, the associate who submitted the review request receives an email and can review the details. The email also contains a “Continue” button which opens the form in a new browser tab. After corrections are submitted, the supervisor receives another email and the workflow repeats itself.

When approved, the Ethics Disclosure Committee is notified by sending a group email within the script.


4. Saving Workflow History

Since history records existed in their original spreadsheet form and they wanted to insert these records into the new work flow spreadsheet as a one-time task, an Apps Script was used to copy the data.  Of course their columns did not match the new spreadsheet.  By using a mapping approach and a “read once” and “write once” technique, the Apps Script quickly made the changes.


function myFunction() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName(Sheet 1);
var rowsWithData = sheet.getRange(2, 1, sheet.getLastRow(),
sheet.getLastColumn()).getValues()
var sheet1Data = [];
var sheet2Data = [];
for (var i = 0; i < rowsWithData.length; i++) {
switch (rowsWithData[i][4]) // This is the activity type
{
...
case "Gift":
sheet1Data.push([rowsWithData[i][12], rowsWithData[i][13],
rowsWithData[i][14]]);
sheet2Data.push([rowsWithData[i][15]]);
continue;
...
default:
continue;
}
}
sheet.getRange(2, 6, sheet1Data.length, 3).setValues(sheet1Data);
sheet.getRange(2, 12, sheet2Data.length, 1).setValues(sheet2Data);
}

Google Apps Script is very powerful and Dito uses it to build interesting solution for its customers. If you are using Google Apps then be sure to use Google Apps Script. You’ll be amazed with what you can build with it.




Steve Webster   profile

Google Sites and Scripts expert from Dito specializing in training and application development. When not busy finding solutions to enhance customer capability in Google Apps, Steve shares examples of his work in the Google Apps Developer Blog.

Read more »

Wednesday, March 11, 2015

4 ways to do Mail Merge using Google Apps Script

Update (August 2014): Try the Yet Another Mail Merge add-on for Google Sheets.

Editor’s Note: This blog post is co-authored by James, Steve and Romain who are Google Apps Script top contributors. -- Ryan Boyd

The Google Apps Script team is on a roll and has implemented a ton of new features in the last few months. Some of us “Top Contributors” thought it will be a useful exercise to revisit the Mail Merge use case and discuss various ways in which we can do Mail Merge using Apps Script. Below are several techniques that tap into the power of Google Apps Script by utilizing Gmail, Documents and Sites to give your mailings some zing. Mail Merge is easy and here is how it can be done.

1. Simple Mail Merge using a Spreadsheet

The Simple Mail Merge tutorial shows an easy way to collect information from people in a Spreadsheet using Google Forms then generate and distribute personalized emails. In this tutorial we learn about using “keys,” like ${"First Name"}, in a template text document that is replaced by values from the spreadsheet. This Mail Merge uses HTML saved in the “template” cell of the spreadsheet as the content source.

2. Mail Merge using Gmail and Spreadsheet Services

The Gmail Service is now available in Google Apps Script, allowing you to create your template in Gmail where it is saved as a draft. This gives us the advantage of making Mail Merge more friendly to the typical user who may not know or care much about learning to write HTML for their template. The mail merge script will replace the draft and template keys with names and other information from the spreadsheet and automatically send the email.

To use this mail merge, create a new spreadsheet, and click on Tools > Script Gallery. Search for “Yet another Mail Merge” and you will be able to locate the script. Then, click Install. You’ll get two authorization dialogs, click OK through them. Add your contact list to the spreadsheet, with a header for each column. Then compose a new mail in Gmail. Follow this syntax for the “keys” in your template: $%column header% (see above). Click Save now to save your draft. Go back to your spreadsheet and click on the menu Mail Merge. A dialog pops up. Select your draft to start sending your emails.

You can add CCs, include attachments and format your text just as you would any email. People enjoy “Inserting” images in the body of their emails, so we made sure to keep this feature in our updated mail merge. To automate this process we will use a new advanced parameter of the method sendEmail, inlineImages. When the script runs it looks in the email template for images and make sure they appear as inline images and not as attachments. Now your emails will look just as you intended and the whole process of mail merge got a whole lot simpler.


3. Mail Merge using Document Forms

The next Mail Merge will use a template that is written in a Google Document and sent as an attachment. Monthly reports, vacation requests and other business forms can use this technique. Even very complex documents like a newsletter or brochure can utilize the automation of Google Apps Script to add the personal touch of having your patron’s name appear as a salutation.

Like in the Mail Merge for Gmail, the Google Docs template will use “keys” as placeholders for names, addresses or any other information that needs to be merged. Google Apps Script can add dynamic elements as well. For example you may want to include a current stock quote using the Financial Service, a chart from the Charts Service, or a meeting agenda automatically fetched for you by the Calendar Service.

As the code sample below demonstrates, the Google Apps Script gets the document template, copies it in a new temporary document, opens the temp document, replaces the key placeholders with the form values, converts it to PDF format, composes the email, sends the email with the attached PDF and deletes the temp document.

Here is a code snippet example to get you started. To use this mail merge, create a new spreadsheet, and click on Tools > Script Gallery. Search for “Employee of the Week Award” and you will be able to locate the script.

// Global variables 
docTemplate = “enter document ID here”;
docName = “enter document name here”;

function sendDocument() {
// Full name and email address values come from the spreadsheet form
var full_name = from-spreadsheet-form
var email_address = from-spreadsheet-form
// Get document template, copy it as a new temp doc, and save the Doc’s id
var copyId = DocsList.getFileById(docTemplate)
.makeCopy(docName+ for +full_name)
.getId();
var copyDoc = DocumentApp.openById(copyId);
var copyBody = copyDoc.getActiveSection();
// Replace place holder keys,
copyBody.replaceText(keyFullName, full_name);
var todaysDate = Utilities.formatDate(new Date(), "GMT", "MM/dd/yyyy");
copyBody.replaceText(keyTodaysDate, todaysDate);
// Save and close the temporary document
copyDoc.saveAndClose();
// Convert temporary document to PDF by using the getAs blob conversion
var pdf = DocsList.getFileById(copyId).getAs("application/pdf");
// Attach PDF and send the email
MailApp.sendEmail(email_address, subject, body, {htmlBody: body, attachments: pdf});
// Delete temp file
DocsList.getFileById(copyId).setTrashed(true);
}

4. Mail Merge using Sites and Spreadsheet Services

For the last example let’s assume you have a great Google Site where you create new letters for your followers. However, you have had some feedback suggest that while many users don’t mind visiting your site, some would prefer to have the newsletter emailed to them. Normally this would require copying and pasting into an email or doc. Why not simply automate this with Google Apps Script?

The body section of a site, the part you edit, can be captured as HTML by the Sites Service and placed in the body of an email. Because the return value is HTML, the pictures and text formatting come through in the email.

Here is a simple example for you to try out:

function emailSiteBody() {  
var site = SitesApp.getPageByUrl(YourPageURL);
var body = site.getHtmlContent();

MailApp.sendEmail(you@example.com, Site Template, no html :( , {htmlBody: body});
}

It really is that simple. Add a for loop with email values from a spreadsheet and this project is done.

Happy merging!

Updated 10/28: fixed instructions for accessing the complete script source for solution 3.




James Ferreira   profile

Author, Scripter, and developer of free apps for non-profits and schools, James has written software to help more than half a million people by extending Google Apps.


Steve Webster   profile

Google Sites and Scripts expert from Dito specializing in training and application development. When not busy finding solutions to enhance customer capability in Google Apps, Steve shares examples of his work in the Google Apps Developer Blog.


Romain Vialard   profile | YouTube

Google Apps Change Management consultant at Revevol, Romain writes scripts to automate everyday tasks, add functionality and facilitate rapid adoption of cutting edge web infrastructures.

Read more »

Building applications on top of Google Apps


Editors Note: This post was written by Michael Cohn and Steve Ziegler from Cloud Sherpas, a software development and professional services company. SherpaTools™ for Google Apps extends Google Apps capabilities. We invited Cloud Sherpas to share their experiences building an application on top of Google Apps utilizing some of our APIs. Cloud Sherpas will also be participating in the Developer Sandbox at Google I/O this May where theyll be demoing their use of Google technologies and answering questions from attendees.

The Directory Manager module of SherpaTools allows administrators to easily manage User Profiles and Shared Contacts in the Google Apps directory. SherpaTools is built on Google App Engine (GAE) utilizing the Google Web Toolkit (GWT), and makes heavy use of the Google Apps Management and Application APIs to provide new administrator and end-user features.

Some customers have tens of thousands of member accounts in their domain. SherpaTools Directory Manager makes it easy to retrieve, edit, and manage user accounts. It also allows you to import or export data in bulk from a Google Docs spreadsheet. This post explains how Directory Manager works with Google Apps, and how we built it. 


Authentication to SherpaTools is achieved by using Google Apps as the OpenID identity provider.  If a user is already logged into Google Apps, they can access SherpaTools without ever providing SherpaTools their credentials.  This Single Sign-On experience greatly enhances user adoption and provides an added security benefit.  If the user is not already logged into Google Apps, they are routed to a Google login page.  With OpenID, SherpaTools never handles the users credentials.


Once logged in, SherpaTools securely requests authorization from Google Apps using 2-legged OAuth (2LO) to make API service calls on behalf of the user.  Since the app has both an end-user and administrator view, it first retrieves the logged in users information from Google Apps via a 2LO-authorized call to the UserService of the Provisioning API.  Depending on the information sent in the API response, the user is either presented with the administrator application or the end-user screen.

Two-legged OAuth (2LO) allows 3rd-party applications like SherpaTools to make authorized API calls to Google Apps on behalf of a user. Here is how we set up our Google Data API ContactsService that will be fetching User Profiles to use 2LO: 

ContactsService contactsService =
    new ContactsService(GlobalConstants.APPLICATION_NAME);
GoogleOAuthParameters parameters = new GoogleOAuthParameters();
parameters.setOAuthConsumerKey(GlobalConstants.CONSUMER_KEY);
parameters.setOAuthConsumerSecret(GlobalConstants.CONSUMER_SECRET);
OAuthHmacSha1Signer signer = new OAuthHmacSha1Signer();
try {
   contactsService.setOAuthCredentials(parameters, signer);
} catch (OAuthException e) {
   // not expected if secret is up-to-date
}
As long as our key/secret pair is correct and the Google Apps customer has entitled our OAuth key to have access to their Contacts API feed, Google authorizes SherpaTools to continue to make API calls.  There are two other settings that should be mentioned in configuring the service to work well on GAE.  First, since we are dealing with somewhat sensitive data, all calls to Google Apps are made over SSL.  To ensure this, we simply set the useSSL flag for the contacts service.  Next, the default request/response timeout on GAE for these API calls is only five seconds out of a possible ten.  Since we will be retrieving as much data as we can within that ten second window to reduce the total number of operations to complete the work, we raise our connection timeout up to just short of that maximum, 9500 milliseconds:
contactsService.useSsl();
contactsService.setConnectTimeout(9500);
Cloud Sherpas embraced a number of Google Web Toolkit best practices to ensure scalability of SherpaTools.  For example, once the app determines which screen the user should see, SherpaTools employs GWT CodeSplitting to optimize and reduce the amount of javascript that needs to be downloaded by the browser client.  The app also uses the GWT RPC framework designed according to the command pattern to transparently communicate with the server, and was architected using the model-view-presenter (MVP) design pattern to allow multiple developers to work on the app simultaneously.


After a Google Apps administrator logs into SherpaTools for the first time, the app caches some key information for better performance.  For example, to populate the User Profile and Shared Contacts lists, the app retrieves the IDs and names of all contacts using the User Profiles API and Shared Contacts API respectively, and writes this information to the data store and memcache.  And for domains with large data sets, SherpaTools uses task queues to break up operations into smaller chunks. 


Since we have to scale the export to handle the contact information of tens of thousands of contact entries, there is no way we can retrieve all of those entries in one request.  Retrieval of this set of information requires breaking the operation up into smaller sub-tasks.  Fortunately, both the GAE Task Queue API and the Google Datastore APIs make it easy to divide the retrieval into smaller chunks.  

GAE Task Queues enable background queueing of HTTP operations (GET, POST, etc.) to arbitrary URLs within our application.  Each of these queued operations are subject to the same restrictions of any other GAE HTTP operation request.  Of particular note is the aforementioned 10 second window to perform our remote service call and the overall 30 second window to complete the total work within a task request. Also, since the Task Queue works from the same set of URLs as are made available to the rest of our application, we need to make sure that there is no ability for unwanted external attempts to execute tasks.  We followed the recommended way of eliminating this possibility by restricting outside access to just our applications admins. 


This constraint restricts all urls starting with /task/ to only be accessible either from system calls such as from the Task Queue or by admins.  The NONE transport guarantee is also important to mention.  We initially attempted to encrypt our task calls using SSL with a transport guaranteed of CONFIDENTIAL, but, at the time we attempted this, execution ceased to function properly.  Since all of the traffic of all of these calls are strictly on Googles internal network we had no issue with making these calls without SSL.
Now that we have our tasks properly secured, we can create a method for sending our User Profiles fetch task request to the Task Queue: 
public void fetchUserProfilesPageTask(String spreadsheetTitle,
    String loggedInEmailAddress, String nextLink, String memcacheKey) {
  Queue queue = QueueFactory.getQueue(USER_PROFILES_QUEUE);
  TaskOptions options =
      TaskOptions.Builder.url("/task/"+USER_PROFILES_FETCH_URL);
  options.param("spreadsheetTitle", spreadsheetTitle);
  options.param("loggedInEmailAddress", loggedInEmailAddress);
  options.param("nextLink", nextLink);
  options.param("memcacheKey", memcacheKey);
  queue.add(options);
}
The url points to a Java HttpServlet that handles the tasks HTTP POST, parses the sent parameters, and calls a method to perform the work:
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
    String loggedInEmailAddress = req.getParameter("loggedInEmailAddress");
    String spreadsheetTitle = req.getParameter("title");
    String nextLink = req.getParameter("nextLink");
    String memcacheKey = req.getParameter("memcacheKey");
    // do the work:
    fetchUserProfilesPage(spreadsheetTitle, loggedInEmailAddress, nextLink, memcacheKey);
}

Summary

This post explains how SherpaTools Directory Manager uses Two Legged OAuth for authentication, and GAE Task Queue API and the Google Datastore APIs to make it easy to divide large retrievals over long intervals into smaller chunks. Other long-running, divisible operations, can use this same approach to spread work across a string of tasks queued in the GAE Task Queue. We would love to hear what you think of this approach and if you have come up with your own solution for similar issues.


Next week we will discuss how we used the User Profile API to retrieve the data sets and the Document List Data API to populate a Google Spreadsheet.


Thanks to the Cloud Sherpas team for authoring this post. Check out SherpaTools at www.sherpatools.com

Read more »

Crowd Sourcing with Google Forms and Fusion Tables

Crowd sourcing has been growing substantially in popularity. More and more businesses and individuals are interested in gathering data from the general public for real-time data analysis and visualization. The concept is being adopted in several fields, including journalism, public health and safety, and business development. During this election year, for example, a journalist might be interested in learning what candidate his or her readers support, and the reasons why they support this candidate.

Google Forms, Fusion Tables, and Apps Script make both data collection and analysis super simple! Using Google Forms, a journalist can quickly create an HTML form for readers to submit their opinions and feedback. Fusion Tables make data analysis easy with several cool data visualization options. Apps Script acts as the glue between Google Forms and Fusion Tables, enabling the Form to send data directly to Fusion Tables.

Let’s take a look at how our journalist friend would use all these tools to collect her reader’s candidate preferences.

Google Forms

Google Forms provides a simple UI tool to develop forms perfect for collecting data from readers. Here’s an example of a simple form the journalist can create to get information from her readers:

Once the form has been created, it can be embedded directly into the journalist’s website or blog using the embeddable HTML code provided by Google Forms.

Google Fusion Tables

Google Fusion Tables makes data analysis simple with its visualization capabilities. Using Fusion Tables, the journalist can create maps and charts of the collected data with just a few clicks of the mouse!

Using some fake data as an example, here’s a pie chart that can be created using Fusion Tables to show the the results of the survey:

With Fusion Tables, it’s also easy to filter data and create a pie chart visualization showing why people like Mitt Romney:

These visualizations can also be embedded in the journalist’s website or blog, as Fusion Tables provides embeddable HTML code for all its visualizations. Now, any time someone visits the webpage with the embedded visualization, they will see the current poll result!

Apps Script

Finally, Apps Script acts as the glue between the Google Form and the Fusion Table, since there is currently no direct way to send Google Form submissions to a Fusion Table. During a hack event last year, I took some time to write an Apps Script script that submits the form data to Fusion Tables. The script uses the onFormSubmit Apps Script functionality as described in this blog post. The Fusion Tables code is based on the code described in this blog post.

To learn how to set up your own Google Form to collect data and save that data in a Fusion Table, please see these instructions.


Kathryn Hurley profile

Kathryn is a Developer Programs Engineer for Fusion Tables at Google. In this role, she helps spread the word about Fusion Tables by presenting at conferences and developer events. Kathryn received an MS in Web Science from the University of San Francisco. Prior work experience includes database management, web production, and research in mobile and peer-to-peer computing.

Read more »