Advanced security for Google Drive for Work

Wednesday, July 23, 2014 | 9:10 AM

Labels: ,



Last month we announced Google Drive for Work, which includes advanced Drive auditing to give organizations control, security and visibility into how files are shared. This new security feature helps companies and IT managers protect confidential information and gain insights into how their employees work.

Drive audit helps IT admins view activity on documents, such as uploading and downloading files, renaming files, editing and commenting, and sharing with others. Filters make it easy to sort and find details like IP address, date range, document title and owner’s email address. To make advanced auditing reports easier to manage, admins can set up alerts for important events like files being shared outside the organization.

To help organizations derive even more value from Drive for Work, we’ve been working with partners to give you even more capabilities through the Drive Audit API:

  • Backupify protects your Google Apps data through secure, automatic, daily backup allowing IT users to easily search and restore files with advanced administrative features, safeguarding your business from data loss caused by user errors, malicious deletions, hackers, and app errors. (website, blogpost)
  • BetterCloud, through their flagship cloud management and security tool, FlashPanel, has enhanced their offering through the Audit API to provide additional controls and insight. (website, blogpost)
  • CloudLock, who provides a pure-cloud Data Loss Prevention (DLP) solution for SaaS applications, has released a new version of CloudLock for Google Drive, leveraging the new Google Drive audit APIs, to enable large organizations to extend their enterprise security controls to the cloud. (website, blogpost)
  • SkyHigh for Google Drive delivers Data Loss Prevention (DLP), mobile-to-cloud support, application auditing, data discovery, and anomaly detection without changing the Google Drive experience users love. (website, blogpost)
And this is only the beginning. We invite developers and customers alike to get started with the Audit API to provide additional advanced security solutions for Google Drive. Learn more by visiting developers.google.com.

Google is committed to enabling organizations to be successful by leveraging a large community of ISVs. One of the areas we constantly invest in is our APIs, that allow customers and ISVs to extend the functionality of the Google Apps platform. If you’d like to join our ISV community, check out the developers.google.com site. For a list of ISVs supporting Google Apps, please visit the Google Apps Marketplace.

Upgrade now to Calendar APIv3

Thursday, July 17, 2014 | 11:30 AM

Labels:

Back in 2011, we launched Calendar APIv3, which offers developers several improvements over older versions of the API, including better support for recurring events and lightweight resource representation in JSON.


At that same time, we also announced that the older versions of the API – v1 and v2 – would be entering a three-year deprecation period in order to give developers time to migrate to the new version. Those three years are coming to an end, and on November 17, the v1 and v2 endpoints will be shut down. If you haven’t already done so, you should migrate your application now to APIv3 so that it continues to work after that date (and to start taking advantage of all that the new API offers!).


For additional resources, check out our Migration and Getting started guides. And if you have questions or issues, please reach out to us on StackOverflow.com, using tag #google-calendar.


By Lucia Fedorova, Calendar API Team

Lucia Fedorova is a Tech Lead of the Google Calendar API team. The team focuses on providing a great experience to Google Calendar developers and enabling new and exciting integrations.

Introducing the new Gmail API

Wednesday, June 25, 2014 | 11:00 AM

Labels:

For a while now, many of you have been asking for a better way to access data to build apps that integrate with Gmail. While IMAP is great at what it was designed for (connecting email clients to email servers in a standard way), it wasn’t really designed to do all of the cool things that you have been working on, which is why this week at Google I/O, we’re launching the beta of the new Gmail API.

Designed to let you easily deliver Gmail-enabled features, this new API is a standard Google API, which gives RESTful access to a user’s mailbox under OAuth 2.0 authorization. It supports CRUD operations on true Gmail datatypes such as messages, threads, labels and drafts.

As a standard Google API, you make simple HTTPS calls and get your responses in JSON, XML or Google Protobuf formats. You can also make these calls from standard web languages like Java and Python without using a TCP socket, which means the API is accessible from many cloud environments that couldn’t support IMAP.

In contrast to IMAP, which requires access to all of a user’s messages for all operations, the new API gives fine-grained control to a user’s mailbox. For example, if your app only needs to send mail on behalf of a user and does not need to read mail, you can limit your permission request to send-only.

To keep in sync, the API allows you to query the inbox change history, thereby avoiding the need to do “archaeology” to figure out what changed.

Finally, a huge benefit is speed. While there’s still some tuning to be done (“beta” - remember?), results from our tests and feedback from pre-release developers suggest that the new Gmail API is delivering dramatic performance improvements over IMAP for web application use cases.

Check out the launch video and get started with samples, tutorials, and API references at https://developers.google.com/gmail/api/. We can’t wait to see what you build.

Posted by Eric DeFriez, Gmail Extensibility Team. As a technical lead for Gmail APIs, Eric works to make it easy for developers to build on top of Gmail.

Guest Post: Integrating with Google Drive via a Chrome Web Store App

Tuesday, June 24, 2014 | 2:00 PM

Labels:

This post is prepared by Nina Gorbunova, Teamlab Office Marketing Manager

About Teamlab Personal
Teamlab Personal is a suite of online document editors free for individual use. We've recently implemented two way integration with Google Drive and would like to share our experience.

Why Google Drive integration
Many of our users connect Google Drive to Teamlab, and we wanted to reach more by being in the Chrome Web Store. The availability of Google Drive SDK and Google Drive API helped us fit it all together. We thought: if a user can connect a Google Drive account to Teamlab Personal, why not build a return path? In the eyes of users, it is an enhancement of their Drive accounts. They get an opportunity to process documents using high formatting quality in browser and to make one more step away from desktop apps.

Integration goals
From the technical side, here is what we wanted to do:

  • Integrate Teamlab editors and viewers with Google Drive.
  • Provide co-editing opportunities.
  • Enable file conversion and creating new files in the common Office Open XML format.
  • Enable users to login with Google to use Teamlab Personal.
Five steps to achieve two-way integration
  1. We registered with Google’s developer console, added our project and connected the Drive API and Drive SDK to the app.
  2. Then we needed to decide what scopes our app needed to access the Google Drive API. We chose the minimal set, ample for us to access the files to edit without trespassing the user’s privacy (most users are not likely to provide full access to 3rd party apps)

  3. Because we work with traditional office apps, we chose docx, xlsx and pptx formats as default file extensions for our app. We also added secondary formats: ppt, pps, odp, doc, odt, rtf, txt, xls, csv, ods, mht, html, htm, fb2, epub, pdf, djvu.
  4. The current listing for the pre-existing app, we modified the code and added the following to the manifest: "container" : "GOOGLE_DRIVE","api_console_project_id" : "YOUR_APP_ID". Once a user installs Teamlab Personal app from Chrome Web Store, it automatically connects to their Google Drive account.
  5. Finally, Teamlab Personal uses OAuth 2.0 for authorization and file access. The application processes requests for creating and opening files.
How it works
As soon as you've installed Teamlab Personal from the Chrome Web Store, the integration automatically activates. Now, you can choose the Teamlab icon when creating new and editing the existing documents.


If the user selects the Teamlab editor as the default, .docx, .xlsx and .pptx files are opened in Teamlab automatically. For other documents, we create a copy in Office Open XML format which will be saved on Drive alongside the original.

Requests processing at personal.teamlab.com

When a file is opened or created on Google Drive using the Teamlab Personal application, the handler gets the request with the following parameters: "ids" ("folderId" if case of file creation), "action", "userId" and "code". The "code" parameter is used to get the authentication token via OAuth 2.0 protocol (with the help of the request to https://accounts.google.com/o/oauth2/token with the "client_id", "client_secret", "redirect_ur parameters", and the additional "grant_type=authorization_code" parameter from the developer console). The received token is used in the subsequent requests. The "ids" parameter is the file to be opened identifier which is sent to the https://www.googleapis.com/drive/v2/files/ address in JSON format. The returned "mimeType" and "downloadUrl" are used to get the file content. That's all what is needed to open the document file in Office Open XML format (.docx, .xlsx or .pptx) in Teamlab.

Files in other formats are converted to the corresponding Office Open XML format and a copy is saved to the Drive folder prior to opening. In this case the "downloadUrl" is used to get the original file. The file is saved with the help of the POST request to the https://www.googleapis.com/upload/drive/v2/files address. In this request the "ContentType" is set as "multipart/related; boundary=boundary" and the request body contains the file information placed before the main request content.

Request code:

string CreateFile(Stream content, string fileName, string mimeType, string folderId, string accessToken){
    var bytes = Encoding.UTF8.GetBytes(
        "\r\n--boundary\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{\"title\":\""
        + fileName + "\",\"parents\":[{\"id\":\"" + folderId + "\"}]}"
        + "\r\n--boundary\r\nContent-Type: " + mimeType + "\r\n\r\n");

    var tmpStream = new MemoryStream();
    tmpStream.Write(bytes, 0, bytes.Length);
    content.CopyTo(tmpStream);

    bytes = Encoding.UTF8.GetBytes("\r\n--boundary--\r\n");
    tmpStream.Write(bytes, 0, bytes.Length);

    var request = WebRequest.Create("https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart");
    request.Method = "POST";
    request.Headers.Add("Authorization", "Bearer " + accessToken);
    request.ContentType = "multipart/related; boundary=boundary";
    request.ContentLength = tmpStream.Length;

    var buffer = new byte[2048];
    int readed;
    tmpStream.Seek(0, SeekOrigin.Begin);
    while ((readed = tmpStream.Read(buffer, 0, 2048)) > 0) {
        request.GetRequestStream().Write(buffer, 0, readed);
    }
    return request.GetResponse().GetResponseStream();
}

Conclusion
The "Works with Google Drive" label does its magic indeed. We strongly recommend other developers build a Chrome Web Store app, as the results are clear and valuable. We had a high jump in installs (see the graph below) after we completed our integration. Teamlab Personal website traffic doubled and we received more than enough of users’ feedback – great impact for further development.

Google Chrome Web Store Impressions&Installations Statistics. Launch - April.

About the author
Nina started her career at Teamlab in 2011 as an intern. She is now a Senior Marketing Manager at Teamlab Office.

Deprecating Script Gallery in the old version of Google Sheets

Friday, June 20, 2014 | 11:45 AM

Labels:

Recently we launched add-ons for Google Docs and Sheets. Now developers can easily package Apps Script applications as add-ons and distribute these scripts via the add-on store. The add-on store gives developers wider distribution, automatic updates, versioning and is vastly superior to the restrictive script gallery that it was designed to replace.

Starting today, we are deprecating the option for developers to publish to the script gallery. No new gallery submissions will be accepted or approved, but scripts already present in the gallery will remain accessible (via the old version of Sheets).

If you rely on distributing or consuming your script from the script gallery, then please convert your script into an add-on and follow the add-on publication instructions.

A One-Two Punch for Getting Even More out of Google Apps APIs

Monday, June 2, 2014 | 3:36 PM

Labels: ,

Editor’s Note: Guest author Steve Ziegler is a senior architect at BetterCloudArun Nagarajan

FlashPanel, a leading enterprise-grade security and management application for Google Apps, makes use of many Google Apps APIs. Each API comes with its own set of rate-limiting quotas, network traffic behavior and response conditions. The challenge we face is to provide our customers with a stable, consistent experience despite these API behavior differences. Fortunately, Google has provided two simple, yet powerful tools to greatly increase our success rate and reliability.

Controlling Throttle Rates with Guava’s RateLimiter


Google Apps APIs operate with quota rates per second or per day. As such, the FlashPanel development team’s objective is to minimize total operation time of a parallelized set of API-intensive processes while avoiding running into Google’s API quota limits. To accomplish this, we can very easily set up a thread-safe RateLimiter for a specific rate per second for each rate restricted API:

final RateLimiter driveApiRateLimiter = RateLimiter.create(QUOTA_RATE_PER_SECOND);

The Guava RateLimiter allows us to attempt to acquire a permit based on the configured rate per second, block until available, and then take it when available, allowing execution.

To use it within the context of a method, we simply reference the rate limiter before making the API call:

public Result performDriveApiCall(driveApiRateLimiter, otherParams){
        driveApiRateLimiter.acquire(); // blocks according to rate
        // make API call...
}
This provides us with a very easy solution for rate limiting parallelized calls to the same APIs.

Reducing Impact of Network Traffic Issues Through Configured Backoffs


Occasionally, we experience additional network traffic issues despite enforcing rate limits. Google suggests that API clients use an exponential backoff strategy to handle this variety of error response. Typically this category of network issue resolves itself on its own after fractions of a second, but occasionally more time is needed. All we need to do to receive a successful API response is to simply retry the call some number of times, with the interval of time between retries growing exponentially.

Again, Google has made this easy through the Google HTTP Client for Java library’s ExponentialBackOff builder. This class allows us to configure the initial retry time interval, the exponential multiplier, the maximum total time to attempt to retry, and the maximum total time between retries. For example, we could configure a retry to span five minutes, growing with a factor of two, starting at a one second interval, and growing up to a maximum of one minute between retries. An API call with this configuration would retry with the following pattern in terms of seconds between retries:

1, 2, 4, 8, 16, 32, 60, 60, 60

If after this last retry, the API call still was not successful, the failed http response is returned. The code to configure such a strategy using the ExponentialBackOff.Builder reads as follows:

ExponentialBackOff backoff = new ExponentialBackOff.Builder()
    .setInitialIntervalMillis(ONE_SECOND)
    .setMultiplier(2.0)
    .setMaxIntervalMillis(ONE_MINUTE)
    .setMaxElapsedTimeMillis(FIVE_MINUTES)
    .build();

One potential “gotcha” that we’ve seen is if we accidentally slam a particular API with many simultaneous API calls. In this event, not only would each of these API calls fail, but they would also schedule their retry strategy to occur simultaneously. All subsequent retries would end up firing simultaneously, causing the API calls to continue to fail due to excess per second volumes. The ExponentialBackoff class accounts for this by including a randomization factor within our retry logic that allows us to have each simultaneous API call stagger at different intervals.

For example, using our previous backoff but now with randomization, one API call may retry with these intervals:

1.04, 1.9, 4.23, 7.8, etc.

While a second API call would retry with something like these intervals:

.98, 2.04, 4.1, 8.15, etc.

With this randomization, we avoid API call sequencing collision, mitigating our chances of encountering quota related errors. To simply add this type of randomization, we append to our builder:

builder.setRandomizationFactor(RANDOMIZATION_FACTOR);

Once we have our exponential backoff strategy configured, the Google HTTP Client for Java library allows us to instantiate and assign an HttpBackOffUnsuccessfulResponseHandler to an HttpRequest as part of the request’s initialization:

private HttpBackOffUnsuccessfulResponseHandler handler = new HttpBackOffUnsuccessfulResponseHandler(backoff);

public void initialize(HttpRequest request){ 
    request.setUnsuccessfulResponseHandler(handler);
}

Final Thoughts


By restricting our API calls using Google Guava’s easy-to-use RateLimiter, and by employing an exponential backoff strategy using the Google HTTP Client for Java library’s ExponentialBackOff, we are able to significantly reduce the amount of network traffic errors received by our Google Apps API calls, improving FlashPanel’s reliability and the overall user experience.

Steve is Senior Architect at BetterCloud, the makers of FlashPanel, the number one security and management application for Google Apps. Follow BetterCloud on Google+ at plus.google.com/+Bettercloud.

Posted by Greg Knoke, Googler

Find Unanswered Emails with Apps Script

Thursday, May 29, 2014 | 10:00 AM

Labels:

Editor’s Note: Guest author Alex Moore is the CEO of Baydin, an email productivity company. --Arun Nagarajan
As the CEO of an email productivity company, not a day goes by when I don’t learn about a new email pain point. I love solving email problems for our customers, but many of their problems do not lend themselves to a full browser-extension and server solution, like the products we make. Apps Script is perfect for solving some of these problems in a quick, lightweight, customizable way.

The Awaiting Response script is a perfect example of one of these solutions. My friend Matt Galligan, the CEO of Circa, tweeted a few months back that he wanted a way to find all of the messages that he sent that did not receive a reply.

Boomerang, our flagship extension, provides a way to bring a single message back to your attention if it doesn’t get a response. But Boomerang is not designed for this particular issue — to use Boomerang in this way, you’d need to move every message you'd ever sent back to your inbox! Instead, it makes more sense to create a label and use Apps Script to apply it to each of these messages.

The Awaiting Response script searches the Sent folder to identify all messages you sent over the previous week. It then checks each thread to determine if someone else replied to your message. If no one has, the script applies the label AwaitingResponse to the message. You can then easily visit that label to see all those messages in a single glance.

var d = new Date();
d.setDate(d.getDate() - DAYS_TO_SEARCH);
var dateString = d.getFullYe

ar() + "/" + (d.getMonth() + 1) + "/" + d.getDate();
threads = GmailApp.search("in:Sent after:" + dateString);
Apps Script provides access to the full power of Gmail search, right from within your script. This snippet uses Javascript’s Date object to construct a Gmail-formatted search query that finds all of the conversations where you’ve sent a message in the last DAYS_TO_SEARCH days. It then loads the results of that search into an array of Thread objects.

var userEmailAddress = Session.getEffectiveUser().getEmail();
var EMAIL_REGEX = /[a-zA-Z0-9\._\-]+@[a-zA-Z0-9\.\-]+\.[a-z\.A-Z]+/g;

# if the label already exists, createLabel will return the existing label
var label = GmailApp.createLabel("AwaitingResponse");

var threadsToUpdate = [];
for (var i = 0; i < threads.length; i++)
{
  var thread = threads[i];
  var lastMessage = thread.getMessages()[thread.getMessageCount()-1];
  lastMessageSender = lastMessage.getFrom().match(EMAIL_REGEX)[0];
  if (lastMessageSender == userEmailAddress)
  {
    threadsToUpdate.push[thread];
  }
}

label.addToThreads(threads)
And this part of the script is where the heavy lifting happens. We iterate through each message in the list of search results, applying a regular expression to the From header in the message to extract the sender’s email address. We compare the sender’s address to the script user’s email address. If they don’t match, we know someone else sent the last message in the conversation. So we apply the AwaitingResponse label to the conversation. If the script user sent the last message, we simply move along to the next message.

Add in a little bit of glue and a couple configuration options, and you have a flexible, simple script that gives you the superpower of always knowing which messages might need you to check back in.

Matt adapted his own version of the script to run automatically each day and to only apply the label to messages sent more recently than the last week.

He has also set up the script to exclude messages that include labels where, for example, he has already used Boomerang to track the messages for later. It would also be a snap to update the script to handle aliases (for example, if you use your Gmail account to send a message using your corporate email address) or to look for messages that require a reply from you.

You can get the script here. To customize it, just create your own copy and edit it right inside the built-in editor. With Awaiting Response, Apps Script helped us solve a customer problem in about fifteen minutes, without having to build an entire product.

Alex Moore is the CEO of Baydin, an email productivity company. Baydin makes software that combines AI and behavioral science to ease the burden on overloaded emailers, including the popular Boomerang email scheduling extension, which has been downloaded over two million times. When taking a break from his email, Alex makes a chicken florentine that tastes like angels singing. He is a rabid Alabama football fan.

Posted by Louis Gray, Googler