# dplyr compatibility

This vignette is aimed at package authors who need to update their code because of a backward incompatible change to dplyr. We do try and minimise backward incompatible changes as much as possible, but sometimes they are necessary in order to radically simplify existing code, or unlock a lot of potential value in the future.

This vignette starts with some general advice on writing package code that works with multiple version of dplyr, then continues to discuss specific changes in dplyr versions.

## Working with multiple dplyr versions

Ideally, you want to make sure that your package works with both the released version and the development version of dplyr. This is typically a little bit more work, but has two big advantages:

1. It’s more convenient for your users, since they’re not forced to update dplyr if they don’t want to)

2. It’s easier on CRAN since it doesn’t require a massive coordinated release of multiple packages.

To make code work with multiple versions of a package, your first tool is the simple if statement:

    if (utils::packageVersion("dplyr") > "0.5.0") {
      # code for new version
    } else {
      # code for old version
    }

Always condition on `> current-version`, not `>= next-version` because this will ensure that this branch is also used for the development version of the package. For example, if the current release is version “0.5.0”, the development version will be “0.5.0.9000”.

Occasionally, you’ll run into a situation where the `NAMESPACE` has changed and you need to conditionally import different functions. This typically occurs when functions are moved from one package to another. We try out best to provide automatic fallbacks, but this is not always possible. Often you can work around the problem by avoiding `importFrom` and using `::` instead. Do this where possible:

    if (utils::packageVersion("dplyr") > "0.5.0") {
      dbplyr::build_sql(...)
    } else {
      dplyr::build_sql(...)
    }

This will generate an `R CMD check` NOTE (because the one of the functions will always be missing), but this is ok. Simply explain that you get the note because you have written a wrapper to make sure your code is backward compatible.

Sometimes it’s not possible to avoid `importFrom()`. For example you might be importing a generic so that you can define a method for it. In this case, you can take advantage of a little-known feature in the `NAMESPACE` file: you can include `if` statements.

    #' @rawNamespace
    #' if (utils::packageVersion("dplyr") > "0.5.0") {
    #'   importFrom("dbplyr", "build_sql")
    #' } else {
    #'   importFrom("dplyr", "build_sql")
    #' }

## dplyr 0.6.0

### Database code moves to dbplyr

Almost all database related code has been moved out of dplyr and into a new package, dbplyr. This makes dplyr simpler, and will make it easier to release fixes for bugs that only affect databases. If you’ve implemented a database backend for dplyr, please read the backend news on the backend.

Depending on what generics you use, and what generics you provide methods for you, you may need to write some conditional code. To help make this easier we’ve written `wrap_dbplyr_obj()` which will write the helper code for you:

    wrap_dbplyr_obj("build_sql")

    wrap_dbplyr_obj("base_agg")

Simply copy the results of this function in your package.

These will generate `R CMD check` NOTES, so make sure to tell CRAN that this is to ensure backward compatibility.

### Deprecation of underscored `verbs_()`

Because the tidyeval framework allows us to combine SE and NSE semantics within the same functions, the underscored verbs have been softly deprecated.

#### For users of SE_ verbs

The legacy underscored versions take objects for which a `lazyeval::as.lazy()` method is defined. This includes symbols and calls, strings, and formulas. All of these objects have been replaced with quosures and you can call tidyeval verbs with unquoted quosures:

    quo <- quo(cyl)
    select(mtcars, !! quo)

Symbolic expressions are also supported, but note that bare symbols and calls do not carry scope information. If you’re referring to objects in the data frame, it’s safe to omit specifying an enclosure:

    sym <- quote(cyl)
    select(mtcars, !! sym)

    call <- quote(mean(cyl))
    summarise(mtcars, !! call)

Transforming objects into quosures is generally straightforward. To enclose with the current environment, you can unquote directly in `quo()` or you can use `as_quosure()`:

    quo(!! sym)
    #> <quosure>
    #> expr: ^cyl
    #> env:  global
    quo(!! call)
    #> <quosure>
    #> expr: ^mean(cyl)
    #> env:  global

    rlang::as_quosure(sym)
    #> Warning: `as_quosure()` requires an explicit environment as of rlang 0.3.0.
    #> Please supply `env`.
    #> This warning is displayed once per session.
    #> <quosure>
    #> expr: ^cyl
    #> env:  global
    rlang::as_quosure(call)
    #> <quosure>
    #> expr: ^mean(cyl)
    #> env:  global

Note that while formulas and quosures are very similar objects (and in the most general sense, formulas are quosures), they can’t be used interchangeably in tidyeval functions. Early implementations did treat bare formulas as quosures, but this created compatibility issues with modelling functions of the stats package. Fortunately, it’s easy to transform formulas to quosures that will self-evaluate in tidyeval functions:

    f <- ~cyl
    f
    #> ~cyl
    rlang::as_quosure(f)
    #> <quosure>
    #> expr: ^cyl
    #> env:  global

Finally, and perhaps most importantly, strings are not and should not be parsed. As developers, it is tempting to try and solve problems using strings because we have been trained to work with strings rather than quoted expressions. However it’s almost always the wrong way to approach the problem. The exception is for creating symbols. In that case it is perfectly legitimate to use strings:

    rlang::sym("cyl")
    #> cyl
    rlang::syms(letters[1:3])
    #> [[1]]
    #> a
    #>
    #> [[2]]
    #> b
    #>
    #> [[3]]
    #> c

But you should never use strings to create calls. Instead you can use quasiquotation:

    syms <- rlang::syms(c("foo", "bar", "baz"))
    quo(my_call(!!! syms))
    #> <quosure>
    #> expr: ^my_call(foo, bar, baz)
    #> env:  global

    fun <- rlang::sym("my_call")
    quo((!!fun)(!!! syms))
    #> <quosure>
    #> expr: ^my_call(foo, bar, baz)
    #> env:  global

Or create the call with `call2()`:

    call <- rlang::call2("my_call", !!! syms)
    call
    #> my_call(foo, bar, baz)

    rlang::as_quosure(call)
    #> <quosure>
    #> expr: ^my_call(foo, bar, baz)
    #> env:  global

    # Or equivalently:
    quo(!! rlang::call2("my_call", !!! syms))
    #> <quosure>
    #> expr: ^my_call(foo, bar, baz)
    #> env:  global

Note that idioms based on `interp()` should now generally be avoided and replaced with quasiquotation. Where you used to interpolate:

    lazyeval::interp(~ mean(var), var = rlang::sym("mpg"))

You would now unquote:

    var <- "mpg"
    quo(mean(!! rlang::sym(var)))

See also `vignette("programming")` for more about quasiquotation and quosures.

#### For package authors

For package authors, rlang provides a compatibility file that you can copy to your package. `compat_lazy()` and `compat_lazy_dots()` turn lazy-able objects into proper quosures. This helps providing an underscored version to your users for backward compatibility. For instance, here is how we defined the underscored version of `filter()` in dplyr 0.6:

    filter_.tbl_df <- function(.data, ..., .dots = list()) {
      dots <- compat_lazy_dots(.dots, caller_env(), ...)
      filter(.data, !!! dots)
    }

With tidyeval, S3 dispatch to the correct method might be an issue. In the past, the genericity of dplyr verbs was accomplished by dispatching in the underscored versions. Now that those are deprecated, we’ve turned the non-underscored verbs into S3 generics.

We maintain backward compatibility by redispatching to old underscored verbs in the default methods of the new S3 generics. For example, here is how we redispatch `filter()`:

    filter.default <- function(.data, ...) {
      filter_(.data, .dots = compat_as_lazy_dots(...))
    }

This gets the job done in most cases. However, the default method will not be called for objects inheriting from one of the classes for which we provide non-underscored methods: `data.frame`, `tbl_df`, `tbl_cube` and `grouped_df`. An example of this is the `sf` package whose objects have classes `c("sf", "data.frame")`. Authors of such packages should provide a method for the non-underscored generic in order to be compatible with dplyr:

    filter.sf <- function(.data, ...) {
      st_as_sf(NextMethod())
    }

If you need help with this, please let us know!

### Deprecation of `mutate_each()` and `summarise_each()`

These functions have been replaced by a more complete family of functions. This family has suffixes `_if`, `_at` and `_all` and includes more verbs than just `mutate` `summarise`.

If you need to update your code to the new family, there are two relevant functions depending on which variables you apply `funs()` to. If you called `mutate_each()` without supplying a selection of variables, `funs` is applied to all variables. In this case, you should update your code to use `mutate_all()` instead:

    mutate_each(starwars, funs(as.character))
    mutate_all(starwars, funs(as.character))

Note that the new verbs support bare functions as well, so you don’t necessarily need to wrap with `funs()`:

    mutate_all(starwars, as.character)

On the other hand, if you supplied a variable selection, you should use `mutate_at()`. The variable selection should be wrapped with `vars()`.

    mutate_each(starwars, funs(as.character), height, mass)
    mutate_at(starwars, vars(height, mass), as.character)

`vars()` supports all the selection helpers that you usually use with `select()`:

    summarise_at(mtcars, vars(starts_with("d")), mean)

Note that intead of a `vars()` selection, you can also supply character vectors of column names:

    mutate_at(starwars, c("height", "mass"), as.character)