CHUVASH.eu

CHunky Universe of Vigourous Astonishing SHarepoint :)

Tag Archives: REST

Using CAML with SharePoint REST API

Do you prefer REST over CSOM as I do? I’ll skip the whys. Andrew Connell put it already in wrtiting so nicely. Well, if you do prefer REST, then you must have discovered some shortcomings of REST, or its incompleteness compared to CSOM. I think of:

  1. Inability to filter items based on multivalued taxonomy fields
  2. Inability to filter items based on user fields where user is added through a group, rather than directly, e.g. AssignedTo=[Me] combined with a SharePoint group.

In such situations I was forced to use CSOM. Until yesterday. Yesterday I learned that we can actually use CAML queries in REST requests.

 

This enables using REST in all situations. The REST API is still developed and many features are added. Maybe a particular operation that needs a CAML query today, can be supported in the core REST API and can be easily refactored then.

But until then, we can use CAML queries in REST requests. Here are the important things about it:

  • A REST request with a CAML query is always a POST request
  • A REST request with a CAML query has always to have X-RequestDigest http header (actually because it is a POST request)
  • A REST request with a CAML query should always have the attached CAML query in the request body (and not in a query string). We don’t want to mess with long urls, do we?
  • A REST request with a CAML query must have the http header “Content-Type: application/json;odata=verbose” unless you use xml in the request body.
Needed HTTP Headers in REST requests

HTTP headers you have to provide in REST requests with CAML queries

You can use jQuery or SP.RequestExecutor to make an ajax call. The REST endpoint is:

_api/web/Lists/GetByTitle('<your list title>')/GetItems

The request body (if you use json, and I bet, you do) is in this format:

{ "query" :
   {"__metadata": 
      { "type": "SP.CamlQuery" }
      , "ViewXml": "<YOUR CAML QUERY>" 
   }
}

Here is the boilerplate for a REST request with a CAML Query:

function getDataWithCaml(listName, caml) {
    var endpoint = "/_api/web/lists/GetByTitle('" 
        + listName + "')/GetItems";
    var requestData = { "query" :
           {"__metadata": 
              { "type": "SP.CamlQuery" }
              , "ViewXml": caml
           }
        };
    return jQuery.ajax({
        url: endpoint,
        method: "POST",
        data: requestData,
        headers: {
            "X-RequestDigest": $("#__REQUESTDIGEST").val(),
            "Accept": "application/json; odata=verbose",
            "Content-Type": "application/json; odata=verbose"
        }
    });
}

This function is just an example. It has no error handling, and it takes for granted that your list is in the root site for on your (sub-)domain (“/”). So take it as an example only.

Here is how the function can be invoked

var caml = "<View><Query><Where><Or><Eq><FieldRef Name='AssignedTo' /><Value Type='Integer'><UserID/></Value></Eq><Membership Type='CurrentUserGroups'><FieldRef Name='AssignedTo' /> </Membership></Or></Where></Query></View>";
getDataWithCaml("Tasks", caml);
Advertisements

Apps can only call the OOB CSOM and REST endpoints

As a SharePoint architect or a SharePoint developer, you must have been thinking about the benefits/limitations of SharePoint apps a lot. I want to point out one of them today, which is very important: using custom webservices deployed to SharePoint inside apps. That is impossible and it is designed to be so due to the security architecture in the sharepoint app framework.

I have read much about SharePoint apps (books, whitepapers, blog posts) and stumbled over these two contradictive statements:

[…] app authenticatton is supported only for scenarios in which an app is calling to the SharePoint host environment by using client-side object model (CSOM) or the REST API. SharePoint 2013 does not support app authentication in any other endpoints beyond these. This means it is not possible to develop and deploy a set of custom web service entry points that support app authentication.

Microsoft SharePoint 2013 App Development, Microsoft Press, page 88.

This is really important to know, because it is exactly the opposite to what it is said in a Microsoft WhitePaper, where we are requested to “some outside-of-the-box thinking”:

The main reason why you would still use full-trust solutions is that a feature or API you want to use is not yet available through the REST endpoints or CSOM APIs. However, this doesn’t mean you can’t still use the server object model from an app that is on-premises. It just requires some outside-of-the-box thinking. Instead of writing a full-trust solution that completely covers the entire business scenario you are trying to satisfy, write a full-trust solution that exposes the functionality you are looking for as a REST endpoint and write an app that can use that endpoint. This will allow your solution to scale while reducing the overall exposure of full-trust code to the SharePoint environment.

Deciding between apps for SharePoint and SharePoint solutions, Microsoft. Keenan Newton.

Just to clearify, the first statement is correct. The second is unfortunately not correct. To be sure I created a web service which I deployed as a farm solution. And when I tried to invoke this webservice from my app I got 403 error and this message

The endpoint /testapp1/_vti_bin/mywebservice.svc
 is not accessible in the context of a SharePoint App.

The endpoint cannot be added within the manifest neither:
endpoint-001

Summary

While developing SharePoint apps we are restricted to the OOB CSOM and REST endpoint only.

OOB here means out-of-the-box, not outside-of-the-box.

REST API: Add a plain text file as an attachment to a list item

plain-text-attachment
SharePoint 2013 REST API has been enhanced and extended. The old _vti_bin/listdata.svc is still there, but the new api for working with lists and list items is much more and obviously a part of a bigger api: _api/web/lists

Yesterday I saw an interesting question on SharePoint StackExchange:

The instructions in the MSDN resource are not so detailed, the cannot be. The guy who asked the question did as it stood in the examples. But sometimes solutions for SharePoint need some small adjustments 🙂

Here is the simplest code to create an attachment in plain text for a list item 1 in the list called List1 in the root web. That’s it. But it works:

var content = "Hello, this text is inside the file created with REST API";
var digest = $("#__REQUESTDIGEST").val();
var composedUrl = "/_api/web/lists/GetByTitle('List1')/items(1)/AttachmentFiles/add(FileName='readme.txt')";
$.ajax({
    url: composedUrl,
    type: "POST",
    data: content,
    headers: {        
        "X-RequestDigest": digest
    }
})

This example is of course just for demonstration. It uses only hard-coded values. But it shows how simple it is to create a list item attachment using SharePoint 2013 REST API and “upload” plain text asynchronously to the server.

Not only plain text (update 2013-03-05)

Of course, uploading only a plain text isn’t enough. The same person Fedor Shihantsov came with an additional question and solved it: Upload a non-text file. Somehow jQuery ajax didn’t work. SP.RequestExecutor worked:

$(document).ready( dofunc );

function dofunc() {
    var control = document.getElementById("ufile");
    control.addEventListener("change", fdocattach, false);
}

var file;
var contents;

function fdocattach(event) {
    var i = 0,
    files = event.srcElement.files,
    len = files.length;

    for (; i < len; i++) {
        console.log("Filename: " + files[i].name);
        console.log("Type: " + files[i].type);
        console.log("Size: " + files[i].size + " bytes");
    }
        

    if (files.length > 0) {
        file = files[0];
        fileName = file.name;

        var reader = new window.FileReader();
        reader.onload = fonload;

        reader.onerror = function(event) {
            console.error("File reading error " + event.target.error.code);
        };
        reader.readAsArrayBuffer(file);
    }       
    return false;
}

function _arrayBufferToBase64(buffer) {
    var binary = '';
    var bytes = new window.Uint8Array(buffer);
    var len = bytes.byteLength;
    for (var i = 0; i < len; i++) {
        binary += String.fromCharCode(bytes[i]);
    }
    return binary;
}

function fonload(event) {
    contents = event.target.result;
    $.getScript("/_layouts/15/SP.RequestExecutor.js", fonload2);
}

function fonload2() {
    var contents2 = _arrayBufferToBase64(contents);

    var createitem = new SP.RequestExecutor("/");
    createitem.executeAsync({
        url: "/_api/web/lists/GetByTitle('List1')/items(1)/AttachmentFiles/add(FileName='" + file.name + "')",
        method: "POST",
        binaryStringRequestBody: true,
        body: contents2,
        success: fsucc,
        error: ferr,
        state: "Update"
    });

    function fsucc(data) {
        alert('success');
    }

    function ferr(data) {
        alert('error\n\n' + data.statusText + "\n\n" + data.responseText);
    }
}

I found another example of SP.RequestExecutor: Calling SharePoint search using REST (e.g. from JavaScript or an app)

$().SPServices is best for SOAP

SPServices is a great tool, really nice work, Marc Anderson (@sympmarc). It has been there all the time I have developed for SharePoint. But the fact that you work with XML and you must parse the attributes (!) was the reason why I prefered listdata.svc and Client Object Model, where you get objects in JSON or you have a nice API to get objects and their properties. But there is an area where SPServices are really the best tool: SharePoint Web Services which only understand SOAP like SocialDataService.asmx.

Three ways to get list items

Just a little comparison. We’ll retrieve Titles from Tasks list (Oppgaver in nb-no).
listdata.svc

$.getJSON("/_vti_bin/ListData.svc/Oppgaver()?$select=Tittel")
	.done(function(data){
	  var tasks = data.d.results;
		for(var t in tasks) {
			var title = tasks[t].Tittel;
			console.log(title);
		}
	})
	.fail(function() {
		console.log("error");
	});

Pay attention to the field name: Tittel. I think it is the biggest shortcoming of listdata.svc: the fact that we can’t use the internal (static) field names.
Client Object Model

var ctx = SP.ClientContext.get_current();
var web = ctx.get_web();
var taskList = web.get_lists().getByTitle("Oppgaver");
var query = new SP.CamlQuery("<ViewFields><FieldRef Name='Title' /></ViewFields>");
var tasks = taskList.getItems(query);
ctx.load(tasks);
ctx.executeQueryAsync(function() {
	var listEnumerator = tasks.getEnumerator();
	while(listEnumerator.moveNext()) {
		var task = listEnumerator.get_current();
		var title = task.get_item("Title");
		console.log(title);
	}
}, function() {
	console.log("error");
});

SPServices

$().SPServices({
    operation: "GetListItems",
    async: true,
    listName: "Oppgaver",
    CAMLViewFields: "<ViewFields><FieldRef Name='Title' /></ViewFields>",
    completefunc: function (xData, Status) {
        $(xData.responseXML).SPFilterNode("z:row").each(function() {
            console.log( $(this).attr("ows_Title") );        
        });
    }
});
SPServices and SocialDataService

Let’s try to find the count of comments for an url. To do so we must create a SOAP envelope in javascript, send it and handle the response result:

var strUrl = "http://dev/Sider/Help.aspx";
var request = "<soap:Envelope \
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' \
    xmlns:xsd='http://www.w3.org/2001/XMLSchema' \
    xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/'> \
    <soap:Body>    \
      <CountCommentsOnUrl \
          xmlns='http://microsoft.com/webservices/SharePointPortalServer/SocialDataService'> \
        <url>" + strUrl + "</url> \
      </CountCommentsOnUrl> \
     </soap:Body> \
  </soap:Envelope>";

jQuery.ajax({
  type: "POST",
  async: true,
  url: '/_vti_bin/SocialDataService.asmx?op=CountCommentsOnUrl',
  data: request,
  contentType: "text/xml; charset=utf-8",
  dataType: "xml",
  success: function (content, txtFunc, xhr) {
    var count = jQuery(content).find('CountCommentsOnUrlResult').text();
    console.log(count);
  }
});

It works but, we have to handle the creating of SOAP envelope in our javascript code which can be error prone. SPServices provides a nice abstraction layer and handles many scenarios. The same operation in SPServices:

$().SPServices({
  operation: "CountCommentsOnUrl",
  async: true,
  url:"http://dev/Sider/Help.aspx",
  completefunc: function (xData, Status) {
    var result = $(xData.responseXML).find("CountCommentsOnUrlResult")[0];
    console.log($(result).text());
  }
});

In both cases we have to iterate through responseXML. This is a sample responseXML for CountCommentsOnUrl from SocialDataService.asmx:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope
  xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
      <CountCommentsOnUrlResponse
          xmlns="http://microsoft.com/webservices/SharePointPortalServer/SocialDataService">
      <CountCommentsOnUrlResult>1</CountCommentsOnUrlResult>
    </CountCommentsOnUrlResponse>
  </soap:Body>
</soap:Envelope>

Update list items with listdata.svc

In one of my previous posts I showed how to retrieve data from listdata.svc with jQuery $.getJSON method. Getting data is very simple, listdata.svc provides a powerful restful interface with filtering, expanding and other features. Read the best and short introduction to listdata.svc by Suneet Sharma or a more detailed and technical article about filters, functions and operators by Mike Flasko at Microsoft with short examples here. Now back to listdata.svc in the SharePoint environment. Corey Roth provides a good images and sample results. For REST the “http verbs” are very important: GET, POST, PUT, DELETE..

SharePoint RESTful interface defines these verbs like that:

HTTP verb Data operation
GET Retrieve
POST Create
PUT Update (update all fields and use default values for any undefined fields)
DELETE Delete
MERGE Update (update only the fields that are specified and changed from current version)

If you want to dive into http verbs in the listdata.svc, install cURL and follow the awesome tutorial at JustinLee.sg.

See this slideshare presentation for a quick introduction:

Allright, a higher level of retrieving and updating of data from listdata.svc is presented by Lee Richardson. Lee uses live binding datacontext, which gives you to save changes: dataContext.saveChanges. Another approach is Client object model.

Retrieve data

I’ll give a try $.ajax. We’ll see if it works. The goal is to update an existing item. Let’s take a usual Task list. I have a site in Norwegian (nb-NO), so we’ll see how well it is suited for i18n.

To retrieve all uncompleted tasks for “contoso\administrator”, we can put this into the browser address bar:

http://contoso/_vti_bin/ListData.svc/Oppgaver?$filter=StatusValue ne 'Fullført' and TilordnetTil/Konto eq 'contoso\administrator'

The invoke is case insensitive. This uses two filters.

To invoke this method with ajax, paste it into $.getJSON:

$.getJSON("http://contoso/_vti_bin/ListData.svc/Oppgaver?$filter=StatusValue ne 'Fullført' and TilordnetTil/Konto eq 'contoso%5cAdministrator'",
{}, null
);

Pay attention to backslash in account name. It must be endoded %5c so javascript doesn’t escape it. In this post I won’t provide any gui for this. The result can be inspected in Chrome Dev Tools Network tab.

Create a new item

To create an item, we have to use the POST verb and send a JSON serialized data in a body.

 //create
var url = "/teamsite/_vti_bin/ListData.svc/MyContacts";
var contact = {
	FirstName: "Arnie",
	Title: "Dell",
	WorkCity: "Lund"
};
var body = JSON.stringify(contact);
$.ajax({
	 type: "POST",
	 url: url,
	 contentType: 'application/json',
	 processData: false,
	 data: body,
	 success: function () {
	   alert('Contact Saved.');
	 }
});

Update an item

As described in the Microsoft resource, to update an existing item from javascript, the best verb is POST with MERGE settings. Provide this js function:

beforeSendFunction = function (xhr) {
  xhr.setRequestHeader("If-Match", "*");
  // Using MERGE so that the entire entity doesn't need to be sent over the wire.
  xhr.setRequestHeader("X-HTTP-Method", 'MERGE');
}

And use it in the $.ajax invoke:

 $.ajax({
 	type: "POST",
	...
        beforeSend: beforeSendFunction,
 });

But before we can invoke $.ajax, we must prepare some properties which will be changed. Let’s change Tittel (Title in nb-NO):

var mods = {};
mods.Tittel = "hello verden"
var body = Sys.Serialization.JavaScriptSerializer.serialize(mods);

Now we can post it:

$.ajax({
 	type: "POST",
	contentType: "application/json; charset=utf-8",
	processData: false,
        beforeSend: beforeSendFunction,
 	url: "/_vti_bin/ListData.svc/Oppgaver(3)",
 	data: body,
 	dataType: "json",
	success: function() { console.log("success"); },
	error: function() { console.log("error"); }
 });

And it works!

The beforeSendFunction can be replaced with headers:

//update 
var url = "/teamsite/_vti_bin/ListData.svc/MyContacts(3)";
var mods = {
	FirstName: "Tolle"
};
var body = JSON.stringify(mods);
//update, another example
$.ajax({
 	type: "POST",
	contentType: "application/json; charset=utf-8",
	processData: false,
        headers: {
		"If-Match": "*",
		"X-HTTP-Method": "MERGE"
	},
 	url: url,
 	data: body,
 	dataType: "json",
	success: function() { console.log("success"); },
	error: function() { console.log("error"); }
 });

To update the status is easy too:

var mods = {};
mods.StatusValue = "Fullført"
var body = Sys.Serialization.JavaScriptSerializer.serialize(mods);
$.ajax({
 	type: "POST",
	contentType: "application/json; charset=utf-8",
	processData: false,
        beforeSend: beforeSendFunction,
 	url: "/_vti_bin/ListData.svc/Oppgaver(3)",
 	data: body,
 	dataType: "json",
	success: function() { console.log("success"); },
	error: function() { console.log("error"); }
 });

To create a new item we’ll use POST as well, don’t provide beforeSendFunction:

var mods = {};
mods.Tittel = "david";
mods.TilordnetTilId = 14;
var body = Sys.Serialization.JavaScriptSerializer.serialize(mods);

 $.ajax({
 	type: "POST",
	contentType: "application/json; charset=utf-8",
	processData: false,
 	url: "/_vti_bin/ListData.svc/Oppgaver",
 	data: body,
 	dataType: "json",
	success: function() { console.log("success"); },
	error: function() { console.log("error"); }
 });

Unfortunately I didn’t find any way to set “assigned to” account name, I had to set Account ID (mods.TilordnetTilId = 14).

To delete an item, just use $.ajax and DELETE verb:

$.ajax({
 	type: "DELETE",
	contentType: "application/json; charset=utf-8",
	processData: false,
 	url: "/_vti_bin/ListData.svc/Oppgaver(3)",
 	data: {},
 	dataType: "json",
	success: function() { console.log("success"); },
	error: function() { console.log("error"); }
 });
Localization

What about i18n and localization? It is actually not so generic. One solution could be localizable javascript enums. The fact that the properties are translated can lead to problems. The first time I load the page after deploy, restart or app pool recycling, I get 400 error (bad request). The explanation can be found in the Response:

{
  "error": {
    "code": "",
    "message": {
      "lang": "sv-SE",
      "value": "No property 'TilordnetTilId' exists in type 'Microsoft.SharePoint.Linq.DataServiceEntity' at position 30."
    }
  }
}

It tries to retrieve the Swedish Properties (due my default language in the browser) of the list items. But the list is in Norwegian (nb-NO). In my other browser where I have nb-NO as default language, this problem doesn’t occur. How can we solve this? Sharepoin is too fast here.

And all this stuff: Fullført and other strings that are only valid for one particular language. How to build a solution which can be deployed in different culture environments. The sick is like SPLinq and Localization, the status values are saved as strings in the database, so you can’t use it like enums and so:

An embryo for a possible solution can be using of localization resources like core.resx, where these values are defined:

protected void Page_Load(object sender, EventArgs e)
{
	if (!IsPostBack)
	{
		RegisterLocalizationScript();
	}
}

private void RegisterLocalizationScript()
{
	var completed = SPUtility
		.GetLocalizedString("core", "Tasks_Completed", 
			(uint)CultureInfo.CurrentCulture.LCID);
	var ongoing = SPUtility
		.GetLocalizedString("core", "Tasks_InProgress", 
			(uint)CultureInfo.CurrentCulture.LCID);
	var script = string.Format("completedString = '{0}';ongoingString = '{1}'", 
			completed, ongoing);
	Page.ClientScript.RegisterStartupScript(GetType(), 
		"todo-localized-values", script, true);
}

or:

// this is used to retrieve and update todos with listdata.svc web service
Task = {
	completed: "Fullført", //$Resoures:core.resx, Tasks_Completed
	inProgress: "Pågår" //$Resoures:core.resx, Tasks_InProgress
}
Datetime

To retrieve and order it after creation date add $orderby:

http://contoso/_vti_bin/ListData.svc/Oppgaver?$filter=StatusValue ne 'Fullført' and TilordnetTil/Konto eq 'contoso\administrator'$orderby=Opprettet desc

To filter datetime follow this link:

http://contoso/_vti_bin/ListData.svc/Oppgaver?$filter=Endret+ge+datetime'2011-11-23T00:00:00'

To get all items changed the last week:

var date = new Date();
date.setDate(date.getDate()-7);
var lastWeekISO = date.toISOString();
$.getJSON("http://contoso/_vti_bin/ListData.svc/Oppgaver?$filter=Endret ge datetime'" + lastWeekISO + "'",
{}, null
);

To update/add a date we have to pass an ISO string.

var today =  new Date();
task.Forfallsdato = today.toISOString();
Misc

Only after I wrote this I found a very nice intro to REST by Jan Tielen.

The alternative is to use SPServices or maybe SPRest or SPELL by Christophe by Path To Sharepoint.

The alternative way to retrieve list items and update them is to use Client Object Model (CSOM). One of the advantages of Client Object Model is that you don’t need to care about the localization of column names and status values. You can rely on static names like you do in the Server Object Model when you use CAML.

This image shows differences for retrieving 10 list items using lisdata.svc vs csom:

Вула Чăвашла

VulaCV - Чăвашла вулаттаракан сайт

Discovering SharePoint

And going crazy doing it

Bram de Jager talking Office 365, SharePoint and Azure

My view and thoughts on Productivity and more.

My programming life

and everything in between

SharePoint Development Lab by @avishnyakov

It is a good place to share some SharePoint stories and development practices.

SharePoint Dragons

Nikander & Margriet on SharePoint

Paul J. Swider - RealActivity

RealActivity is a specialized healthcare services and solution advisory firm.

Mai Omar Desouki - Avid SharePointer

Egyptian & Vodafoner - Senior SharePoint Consultant

Cameron Dwyer | Office 365, SharePoint, Outlook, OnePlace Solutions

Office 365, SharePoint, OnePlace Solutions & Life's Other Little Wonders

paul.tavares

Me and My doings!

Share SharePoint Points!!

By Mohit Vashishtha

Jimmy Janlén "Den Scrummande Konsulten"

Erfarenheter, synpunkter och raljerande om Scrum från Jimmy Janlén

SPJoel

SharePoint for everyone

SharePointRyan

Ryan Dennis is a SharePoint Solution Architect with a passion for SharePoint and PowerShell

SharePoint 2020

The Vision for a Future of Clarity

Aharoni in Unicode, ya mama

Treacle tarts for great justice

... And All That JS

JavaScript, Web Apps and SharePoint