Basic

May 26 2016

13 comments

Unity Sprites: SpriteRenderer vs. CanvasRenderer (UI Image)

By Rubén Torres Bonet

May 26, 2016


__CONFIG_colors_palette__{"active_palette":0,"config":{"colors":{"6c4de":{"name":"Main Accent","parent":-1},"67ed2":{"name":"Accent Dark","parent":"6c4de","lock":{"saturation":1}}},"gradients":[]},"palettes":[{"name":"Default","value":{"colors":{"6c4de":{"val":"var(--tcb-skin-color-0)"},"67ed2":{"val":"rgb(59, 65, 63)","hsl_parent_dependency":{"h":160,"l":-0.05,"s":0.04}}},"gradients":[]},"original":{"colors":{"6c4de":{"val":"rgb(51, 190, 127)","hsl":{"h":152,"s":0.57,"l":0.47}},"67ed2":{"val":"rgb(59, 65, 63)","hsl_parent_dependency":{"h":160,"s":0.04,"l":0.24}}},"gradients":[]}}]}__CONFIG_colors_palette__

While working in a project for one of our clients, I was asked about the difference between sprites (SpriteRenderer) and UI images (CanvasRenderer) in Unity. I didn’t find much information about it so I decided to prepare a presentation in my company to help making it clear. In this post you will find a more detailed version of the original slides I prepared for it. This blog entry will be based on Unity 5.3.4f1.

Sprites are basically semi-transparent textures that are imported as sprites in its texture import settings. They are not directly applied to meshes like textures, but rather on rectangles/polygons (at the end, they are meshes too, so not such of a big difference). Sprites are images that will be rendered in either your 2d/3d scene or in your interface.

1. Usage

It is straightforward to use sprites in Unity. Just drop the desired image (in PNG preferably) in the assets folder and click on it to access the inspector settings. Mark it as sprite (2D and UI) as shown in the screenshot below.

Unity sprite import settings

Unity sprite import settings

Now it is time for you to decide between using it as a sprite or as an UI Image. But if you are reading this, you might not be sure which method you want. We will describe the differences in the next section; for now we will just quickly outline how to create both of them.

If you want to use a SpriteRenderer, just drop the sprite from the project view into the hierarchy or scene view. The inspector of the new GameObject will look like:

Unity Sprite Renderer
Unity Sprite Renderer

If you wanted to create a UI Image instead, just right click in the hierarchy and create new UI -> Image. That component requires a canvas, so it will be created if you don’t have one yet. At the end, it will look like:

Unity Image Renderer
Unity Image Renderer

2. Comparison: SpriteRenderer vs CanvasRenderer

When it comes to the hierarchy, you can place sprites wherever you want in your scene. UI Images, on the other hand, have to be inside a canvas (a GameObject having a Canvas component). You can position sprites just like all other objects through its transform, but images will use a RectTransform instead so as to help positioning the image in your interface system.
Sprites are rendered in the transparent geometry queue (unless you use another material than the default). UI Images are also rendered in the transparent geometry queue (Render.TransparentGeometry), unless you use the overlay rendering mode in which case it will be rendered through Canvas.RenderOverlay. As you might have guessed, it is relatively expensive to draw them on mobile. We will explain later why.
One of the key differences between sprites and images is that sprites support the automatic creation of meshes out of it, where UI images consist always of rectangles. The reason for creating meshes will be explained in the next section; we will see how important it is, as it has an important performance impact.
Lastly, both can be used with sprite atlases in order to reduce draw calls. That’s good.
It might help seeing the differences with concrete examples.

Unity Sprite vs. Image - 1

Check the differences between both methods in example 1. UI Image created a tight rectangle that envelops the sprite, whereas the SpriteRenderer created a mesh that better fits the sprite we are rendering. Let’s check another example:

Unity Sprite vs. Image - 2
Unity Sprite vs. Image

Likewise, the same happened in example 2. But the mesh looks much more complicated now, why is that? Unity tries to fit the sprite the best it cans without introducing too many polygons, so that is the result we get. One could argue if the trade-off is beneficial or not. And what happens now, if we import a PNG with islands, e.g. an image containing different figures separated by transparent areas?

Unity Sprite vs Image - 3
Unity Sprite vs Image – 3

Very interesting results in the third example. SpriteRenderer creates two submeshes, one per island; however, UI Image extends the rect so it covers the whole image. Anyway, is this important?

3. Performance

You guessed it, it’s closely related to performance. It does make a huge difference if you are rendering many of them (like grass in a terrain, or particles). So let us analyse the reasons for it.
Whenever you render a texture, you need to send a command to the GPU driver. You set some information, like the vertices, indices, uv coordinates, texture data and shader parameters and then you make the famous draw call. Afterwards, some nasty things happen in the GPU before the final image is displayed in your screen. A (really) simplified rendering pipeline consists of the following:

  1. CPU sends a draw command to the GPU.
  2. The GPU gets all the information it needs for drawing (like copying the textures to its VRAM, if it’s got one).
  3. The geometries sent are transformed into pixels through vertex shaders and the rasterizer.
  4. Every pixel is transformed through fragment shaders and written one or several times into the frame buffer.
  5. When the frame is completed, the image is displayed in your screen so you can have fun playing.

So back to the topic, what is the difference between SpriteRenderers and UI Images? It might seem that sprites are more expensive since its geometry is more complex. But what if I told you that vertex operations are normally cheaper than fragment operations? That is especially true for mobile and semi-transparent objects. But why?
In many engines, including Unity, semi-transparent materials are rendered back-to-front. That means, the farthest object (from the camera) are always drawn first so the alpha blending operations work as expected. For opaque materials it is just the opposite so we can cull objects that are not visible.
Pixel shaders will be executed for every (visible) pixel of the sprite you are rendering, so if you have a huge sprite (in relation to the screen size), then you will execute that fragment shader on many pixels. The problem with transparent objects is that you can not effectively cull them if they lie in your camera frustrum, so basically you are rendering ALL semi-transparent objects even if many of them will not be visible at the end. So you find yourself rendering many times the same pixels and overwriting them the whole time in the frame buffer. This issue is commonly known as overdraw. And yes, it is a problem since it wastes memory bandwith and you will quickly reach the pixel fillrate limit of your GPU, something you can’t allow if you are targeting mobile. That is the key point to be understood.
Now, if you did understand it, you will also figure out why SpriteRenderer and CanvasRenderer are so different. The former creates meshes that eliminates unneeded transparent pixels (therefore avoiding executing expensive fragment shaders and reducing overdraw), while UI Image creates a simple mesh that probably provokes a lot of overdraw. There’s always a balance you have to achieve between having complex geometry or living with more fragment operations.
You should always consider using sprite atlases, since sprites are normally large in number but small in size. That leads to having so many draw calls as sprites, and that is not good. Also, image compression works better if you have larger images.

Unity Atlasing
Unity Atlasing

You can easily create sprite atlas through the Atlas Packer. Still, sometimes the automatically created meshes are not optimal and you have no control over it, so you might consider using more advanced software like ShoeBox or TexturePacker.

Unity Atlasing vs. TexturePacker
Unity Atlasing vs. TexturePacker

4. Conclusion

The next time you work with sprites, think about the following:

  1. If you only have a few sprites, use whatever you prefer. If you have hundreds of them, reread this post.
  2. Use the profiler and the frame debugger to know what is going on.
  3. Avoid using transparency. Try replacing sprites with opaque alternatives if your budget allows it.
  4. Avoid rendering big sprites on screen, as more overdraw will happen. You can easily check for overdraw issues in the scene view by selecting Overdraw as rendering mode. This is critical for particle systems.
  5. Prefer having a more complicated geometry than a lot of pixels, especially on mobile. Check the result in the scene view by rendering it as shaded wireframe.
  6. If you need the interface positioning helpers (like content fitter, vertical groups, etc.) go for UI Images.
  7. To know if you are hitting the pixel fillrate limits, check if performance substantially improves by reducing the resolution of your rendered area.

Let me know if you have questions or would like to point out a mistake. Any other appreciation is also welcome.

  • Interesting article.
    Just because you talked about TexturePacker. I have single images I want in a single spritesheet together, but I need to access to the Texture2D of each single sprite, ’cause I need to access to the pixels with the GetPixel() function.
    I normally do it through the Sprite.texture field but, as the documentation says, that field return the Texture2D ref of the entire spritesheet if my sprite is packed.
    https://docs.unity3d.com/ScriptReference/Sprite-texture.html
    I could create a single Texture2D from the spritesheet but I don’t know if is an efficient and not so heavy operation. Also I was looking for a simple solution, if possible.

    • Hi Francesco,
      AFAIK you can’t have a Texture2D of each single sprite, since you have a full texture that is the atlas itself. That is the reason for having atlases, having a single texture instead of hundreds of smaller textures. You might be able to create a Texture2D out of it, but then you lose all the advantages of texture atlases.
      What you can also do is getting a reference to that sprite and then get its rect, i.e. the area the sprite occupies within the atlas. For that use Sprite.textureRect (http://docs.unity3d.com/ScriptReference/Sprite-textureRect.html). There are similar variations that may be handy: textureRectOffset, rect. With those you may use then GetPixel.
      Regards

      • Thanks Ruben, really hepful. I guess in this way I can’t use the polygonal algorithm in TexturePacker but just che maxrect one. Btw it’s a great step forward than the single sprites solution.
        Regards

  • I still don’t get why we don’t get the option to use tightly-mesh-traced UI elements. Usually UI is rendered at native resolutions (to benefit from that nice crispiness of today’s devices) while 3D can get away with lower resolution.
    Also, UI could have A LOT LESS elements than a full-featured sprite based 2D game, so the cost of using more vertices is a given.
    Any thoughts? Or I’m thinking sideways?

    • Hi Alej.

      I can only guess.

      Batching is way more complex in UI systems, e.g. overlap detection.
      Adding more vertices allows for different shapes. That’d make canvas rebuilds much more expensive.

      That said, one can create its own UI component with custom vertices. Textmesh pro does that.

      Other reason might be that they just didn’t prioritise it.

      What are your thoughts?

  • {"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

    Rubén Torres Bonet

    About the author

    Born Game Developer, now ready to help you develop better games. Primary programmer on Star Trek Bridge Crew (Oculus Quest), Diamond Dash. Programmer on Time Stall, Catan Universe, Anne Frank House VR, Jelly Splash, Blackguards Definitive Edition. I also worked in minor XR experiences for HoloLens and Vive for clients such as Audi and Volkswagen.

    You might also like

    Simplify Your Life With Unity Mesh Simplifier
    >