Bug #1054

Ubuntu 16.04 and mesa drivers

Added by Rob Baer over 2 years ago. Updated about 1 year ago.

Status:Fix exists, needs testingStart date:09/02/2016
Priority:HighDue date:
Assignee:Jonas Hauquier% Done:

0%

Category:OpenGL
Target version:MakeHuman 1.1.2

Description

ubuntu 16.04 MH2073

rwbaer@Aspire:~$ inxi -Gx
Graphics: Card: Intel Haswell-ULT Integrated Graphics Controller
bus-ID: 00:02.0
Display Server: X.Org 1.18.3 drivers: intel (unloaded: fbdev,vesa)
Resolution:
GLX Renderer: Mesa DRI Intel Haswell Mobile
GLX Version: 3.0 Mesa 12.1.0-devel Direct Rendering: Yes

Although the purpose of this is MH testing, I'll also accept rollback/fix advice from any Ubuntu gurus amonst you ;-)

Initially, I "think" Makehuman was rendering fine with Ubuntu and my mesa drivers. [I don't know this for sure because this is hindsight memory.] After an update, it "APPEARS" that the texture is being rendered "twice" once using the proper UV's and the second time without Uvs. Don't know if I can roll back or start over, but I thought the situation was worth documenting.

screenshot_1_1472825166.png (199 KB) Rob Baer, 09/02/2016 04:06 PM

screenshot_1_1473028065.png (368 KB) Rob Baer, 09/05/2016 12:28 AM

screenshot_2_1473028123.png (159 KB) Rob Baer, 09/05/2016 12:29 AM

screenshot_3_1473028211.png (159 KB) Rob Baer, 09/05/2016 12:30 AM

grab_2016-10-10_19.16.37.png (730 KB) Aranuvir #, 10/10/2016 07:20 PM

litsphere_fragment_shader.txt Magnifier - Modified shader to show problem (3 KB) Thanasis Papoutsidakis, 01/25/2017 05:53 PM

ShaderFailLogs.zip (18.5 KB) Rob Baer, 02/26/2017 01:24 AM

1272
1275
1276
1277
1286

Related issues

Related to MakeHuman - Bug #1061: Debug opengl failure Fix exists, needs testing 09/15/2016
Related to MakeHuman - Bug #154: Investigate and fix rendering issues with intel graphics ... Accepted 02/25/2014
Related to MakeHuman - Bug #1106: Glitchy Skin Rejected 01/30/2017
Related to MakeHuman - Feature #1127: Provide users with an easy way to downgrade to --noshaders Fix exists, needs testing 02/16/2017
Related to Python 3 - Bug #1170: Opengl issues on Windows in MakeHuman for Python3 New 04/25/2017
Related to Python 3 - Bug #1203: ubuntu 16.04 intel graphics -- shader update and opengl... New 08/13/2017

History

#1 Updated by Rob Baer over 2 years ago

and for good measure:

rwbaer@Aspire:~$ glxinfo | grep -i opengl
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Haswell Mobile
OpenGL core profile version string: 3.3 (Core Profile) Mesa 12.1.0-devel
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.0 Mesa 12.1.0-devel
OpenGL shading language version string: 1.30
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 12.1.0-devel
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
OpenGL ES profile extensions:

#2 Updated by Aranuvir # over 2 years ago

Currently I can confirm to have the same issue with my Debian testing installation, though didn't do further investigation about it, yet. Before switching to Debian I've been using Ubuntu 15.10 and didn't encounter a similar issue (so, probably not a MakeHuman bug). Perhaps you can try the PPA from xorg-edgers...

#3 Updated by Rob Baer over 2 years ago

The texture repeat problem also occurs with the eyes texture even when the default skin is enabled.

I don't know if this issue points to anything in MH that is uncovered by the new mesa drivers or just points to the mesa driver dysfunction per. How does --noshaders work? Are we getting one rendering with ( a UV map broken shader) and one rendering without a shader superimposed on it? Need to try this with --noshaders flag.

#4 Updated by Rob Baer over 2 years ago

Indeed, using --noshaders does fix the problem. This issue may therefore be a different manifestation the "black and blue" issue that has plagued old Intel cards in the past. Do we have any additional insight on whether the switch helps by turning off a specific shader? Is this related to litsphere shader specifically? Note that this is a relatively new 2 year old card and the GLSL support seems nominally well-beyond the minimum.

#5 Updated by Aranuvir # over 2 years ago

Starting MakeHuman with DRI_PRIME=1 doesn't make a difference in regard of the rendering issue, though the renderer used now is AMD:

GL.VENDOR: X.Org
GL.RENDERER: Gallium 0.4 on AMD PITCAIRN (DRM 2.43.0 / 4.6.0-1-amd64, LLVM 3.8.1

This is what I get when using Intel:

GL.VENDOR: Intel Open Source Technology Center
GL.RENDERER: Mesa DRI Intel(R) Haswell Mobile

#6 Updated by wolgade (Forum User) about 2 years ago

I encountered this issue a while back after upgrading to Ubuntu 15.10. So I don't think it's been introduced with 16.04. This is true at least for Ivy Bridge. See also: http://www.makehumancommunity.org/forum/viewtopic.php?f=3&t=12828

#7 Updated by Rob Baer about 2 years ago

  • Related to Bug #1061: Debug opengl failure added

#8 Updated by Rob Baer about 2 years ago

Great information about the Ubuntu 15.10 and the fact that regressing helped. The question remains as to whether this is a pure openGL version error, or a mixed error involving both MH implementation and openGL versioning. It could be a little difficult to trace down. Pretty sure that the first mesa drivers I had with ubuntu 16.04 worked fine, and the break only happened when I installed updates.

Joel has 16.04, but uses nVidea cards; I think Aranuvir is using AMD. My Ubuntu test machine uses simple built-in Intel drivers.

#9 Updated by Aranuvir # about 2 years ago

  • Category set to OpenGL
  • Status changed from New to Accepted
  • Priority changed from Normal to High
  • Target version set to MakeHuman 1.1.1

#10 Updated by Jonas Hauquier about 2 years ago

The problem here is that the wrong texture is sampled by the litsphere shader. It's sampling the diffuse texture instead of the litsphere map.

#11 Updated by Aranuvir # about 2 years ago

I do not know that much about it, but I thought the lit sphere shader is used when no other texture is chosen. But the issue only occurs when selecting a texture. Looking at the model's right waist of the attached image, one can see a face, this occurs several times on other parts of the model. This leads to the impression of a mapping issue.

#12 Updated by Rob Baer about 2 years ago

Just to reiterate, this is a Linux-specific problem, and rendering of the same textures works fine on Windows and OSX. It could be using the wrong shader, I suppose, but this would need to be happening in an OS and driver specific manner. As Aranuvir points out, it appears that the texture is added to the human twice, once with the proper UV mapping and once with the wrong mapping. The generic, non-texture material looks fine, even on Linux.

Edit: As to the double sampling (mapping?) statement, I'm basing this inference, in part, on the eyes (see comment 3) where the iris seems represented twice - once in its proper position and once on the sclera.

#13 Updated by Jonas Hauquier about 2 years ago

Aranuvir (Forum Moderator) wrote:

I do not know that much about it, but I thought the lit sphere shader is used when no other texture is chosen. But the issue only occurs when selecting a texture. Looking at the model's right waist of the attached image, one can see a face, this occurs several times on other parts of the model. This leads to the impression of a mapping issue.

It's not a mapping issue. It's a wrong sampler issue.

#14 Updated by Thanasis Papoutsidakis almost 2 years ago

Problem persists on weston desktop (wayland), so it seems to not be (Xorg) display-server related.

#15 Updated by Rob Baer almost 2 years ago

Welcome back Thanasis. I don't know how easy the history of this is to read, but the machine for the original report is dual boot Windows 10 and Ubuntu 16.04. The very same images render fine with the same graphics card on the Windows side.

Further, I'm 99% certain that at one time they were rendering fine with Ubuntu 14.10 and as far forward as my initial upgrade to 16.04. The problem emerged in conjunction with an update. My Linux skills are not strong enough to move backwards in time with this, but maybe that catches you up at least. Let me know if you want any testing done.

#16 Updated by Aranuvir # almost 2 years ago

Hi Thanasis, nice to see you back!

Last weekend I had done some tests to narrow down the issue. I have installed a recent Fedora 25. Here the issue does not occur. Obviously they are using an old Intel graphics driver: http://www.phoronix.com/scan.php?page=news_item&px=Fedora-Xorg-Intel-DDX-Switch. It was last summer when I encountered the issue for the first time, probably when Debian/Ubuntu switched to the new graphics driver.

#17 Updated by Rob Baer almost 2 years ago

Interesting find Aranuvir.

Following the links to some related pages, I found this: https://tjaalton.wordpress.com/2016/07/23/intel-graphics-gen4-and-newer-now-defaults-to-modesetting-driver-on-x/ yieled whcih includes the comment:

I’ve seen only one bug filed that was caused by this change so far, and it turned out to be a kernel bug fixed in 4.6 (Yak will ship with 4.8). If you see something strange like corrupt widgets or whatnot after upgrading to current Yakkety, verify it doesn’t happen with intel (‘cp /usr/share/doc/xserver-xorg-video-intel/xorg.conf /etc/X11’ followed by login manager restart or reboot) and file a bug against xserver-xorg-core (verify xdiagnose is installed, then run ‘ubuntu-bug xserver-xorg-core)’. We’ll take it from there.

Does MH need to do something to support this "generic GLAMOR module" or should that be part of the video chain provided by the OS and drivers? Seems like Fedora will soon not be working with MH either if your blog link is accurate.

https://www.freedesktop.org/wiki/Software/Glamor/

#18 Updated by Jonas Hauquier almost 2 years ago

I think the problem is tge opengl or shader code is not 100% standards compliant. Nany implementations are tolerant, but it seems mesa does nit like it.

#19 Updated by Aranuvir # almost 2 years ago

@Rob (slightly of topic): I got the information about Fedora from one of my local computer magazines, though it is very likely they got it from phoronix again. When you are on Linux, this link is for your bookmarks. I suppose the impact factor this site is quite high. :)

@Jonas: I agree that it's probably more an implementation issue rather than a driver issue. Do you think #154 is related?

#20 Updated by Jonas Hauquier almost 2 years ago

Aranuvir (Moderator) wrote:

Do you think #154 is related?

Likely

#21 Updated by Jonas Hauquier almost 2 years ago

  • Related to Bug #154: Investigate and fix rendering issues with intel graphics on windows added

#22 Updated by Thanasis Papoutsidakis almost 2 years ago

Welcome back Thanasis.

Hi Thanasis, nice to see you back!

Hi everyone! :)
I have much more free time now, after finishing some difficult courses in university, so I'm here again.

Now about the bug. I tried browsing the commit log of the mesa project between the versions that were used in the two Ubuntu releases, but these guys release like 100 commits a day and it's practically impossible to spot anything of concern in there. So the other option is trial and observation.

I tried fiddling with the litsphere shader and got some interesting results. At first I commented out the lines that sample the diffuse texture, and as expected the result was a shaded, untextured human. Then I uncommented the lines that loaded the diffuse texture (texture2D()) and the line assigning the alpha channel, excluding the other channels. The result was the human with only the problematic texture applied. It's like the diffuse texture is somehow replacing the litsphere.

I attach the shader with the commented out lines. Notice that if you comment out the alpha assignment and use this one (outColor.a = 1.0;) the problem will dissapear. It's triggered only when we sample the diffuse texture.

I'll keep playing with the shader and keep you updated.

#23 Updated by Aranuvir # almost 2 years ago

Deleted ...

#24 Updated by Aranuvir # almost 2 years ago

Deleted ...

#25 Updated by Aranuvir # almost 2 years ago

Edit 2: I had better checked the error logs before. The change crashed shader.py, with the same result as starting MH with --noshaders, so it just seemed to be solved. F*** ... Sorry again. :(

#26 Updated by Rob Baer almost 2 years ago

#27 Updated by Joel Palmius almost 2 years ago

  • Related to Feature #1127: Provide users with an easy way to downgrade to --noshaders added

#28 Updated by Rob Baer almost 2 years ago

Even understanding the problem is complicated, let alone fixing it.

This is a really nice blog series on 3D graphics pipelines https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-graphics-pipeline-2011-index/ Part 1 has a little subsection called "Small aside: OpenGL" that makes it clear that the platform specificity of our problems is not entirely unexpected.

#29 Updated by Rob Baer almost 2 years ago

Ubuntu 16.04 r2072 (b0f8150c39fe) [This is built in Intel graphics card machine]

I'm hoping this is the same error and not something different, but I finally managed to get a set of dependencies that produces a reproducible openGL error on startup when using the opengl debug switches. Full Log set is attached in zip above.

Does this reveal something important for this error or does it need separate reporting. I have lots of (too much?) studying to do before I can tell what it means all by myself. Anyway, hope it helps us ...

...
[2017-02-25 17:34:00,111] logs.py->__call__():72 -- MESSAGE -- from_param( <class 'OpenGL.arrays.arraydatatype.GLfloatArray'>,array([ 0.], dtype=float32) )
[2017-02-25 17:34:00,120] logs.py->__call__():72 -- MESSAGE -- gluErrorString( 1282 )
[2017-02-25 17:34:00,122] logs.py->__call__():77 -- WARNING -- Failure on glGetUniformfv: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/OpenGL/logs.py", line 74, in __call__
    return function( *args, **named )
  File "/usr/lib/python2.7/dist-packages/OpenGL/error.py", line 208, in glCheckError
    baseOperation = baseOperation,
GLError: GLError(
    err = 1282,
    description = 'invalid operation',
    baseOperation = glGetUniformfv,
    cArguments = (1L, 3, array([ 0.], dtype=float32))
)

[2017-02-25 17:34:00,123] log.py->warning():119 -- WARNING -- Exception during event onStart
Traceback (most recent call last):
  File "./core/events3d.py", line 211, in callEvent
    method(event)
  File "./core/mhmain.py", line 806, in onStart
    self.startupSequence()
  File "./core/mhmain.py", line 728, in startupSequence
    self.loadHuman()
  File "./core/mhmain.py", line 383, in loadHuman
    self.selectedHuman = self.addObject(human.Human(files3d.loadMesh(mh.getSysDataPath("3dobjs/base.obj"), maxFaces = 5)))
  File "./apps/human.py", line 82, in __init__
    self.material = material.fromFile(getSysDataPath('skins/default.mhmat'))
  File "./shared/material.py", line 1372, in fromFile
    mat.fromFile(filename)
  File "./shared/material.py", line 476, in fromFile
    self.configureShading(diffuse=shaderConfig_diffuse, bump=shaderConfig_bump, normal=shaderConfig_normal, displacement=shaderConfig_displacement, spec=shaderConfig_spec, vertexColors=shaderConfig_vertexColors, transparency=shaderConfig_transparency, ambientOcclusion=shaderConfig_ambientOcclusion)
  File "./shared/material.py", line 913, in configureShading
    self._updateShaderConfig()
  File "./shared/material.py", line 950, in _updateShaderConfig
    bump = self._shaderConfig['bump'] and self.supportsBump()
  File "./shared/material.py", line 845, in supportsBump
    if self.shaderObj and result:
  File "./shared/material.py", line 1010, in shaderObj
    return self.getShaderObj()
  File "./shared/material.py", line 1006, in getShaderObj
    return shader.getShader(shaderPath, self.shaderDefines)
  File "./lib/shader.py", line 515, in getShader
    shader = Shader(path, defines)
  File "./lib/shader.py", line 306, in __init__
    self.initShader()
  File "./lib/shader.py", line 433, in initShader
    self.updateUniforms()
  File "./lib/shader.py", line 455, in updateUniforms
    for uniform in self.getUniforms():
  File "./lib/shader.py", line 449, in getUniforms
    uniform.update(self.shaderId)
  File "./lib/shader.py", line 134, in update
    self.glquery(pgm, self.index, values)
  File "/usr/lib/python2.7/dist-packages/OpenGL/platform/baseplatform.py", line 379, in __call__
    return self( *args, **named )
  File "/usr/lib/python2.7/dist-packages/OpenGL/logs.py", line 74, in __call__
    return function( *args, **named )
  File "/usr/lib/python2.7/dist-packages/OpenGL/error.py", line 208, in glCheckError
    baseOperation = baseOperation,
GLError: GLError(
    err = 1282,
    description = 'invalid operation',
    baseOperation = glGetUniformfv,
    cArguments = (1L, 3, array([ 0.], dtype=float32))
)
[2017-02-25 17:34:00,177] logs.py->__call__():72 -- MESSAGE -- glDeleteShader( 2L )
[2017-02-25 17:34:00,179] logs.py->__call__():72 -- MESSAGE -- glDeleteShader( 3L )
[2017-02-25 17:34:00,181] logs.py->__call__():72 -- MESSAGE -- glDeleteProgram( 1L )
...

For the record here's the info on the graphics card is NOW:


rwbaer@Aspire:~$ glxinfo |grep -i opengl
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Haswell Mobile 
OpenGL core profile version string: 4.3 (Core Profile) Mesa 17.1.0-devel - padoka PPA
OpenGL core profile shading language version string: 4.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.0 Mesa 17.1.0-devel - padoka PPA
OpenGL shading language version string: 1.30
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 17.1.0-devel - padoka PPA
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
OpenGL ES profile extensions:
rwbaer@Aspire:~$

As compared to BEFORE when I opened this issue:

rwbaer@Aspire:~$ glxinfo | grep -i opengl
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Haswell Mobile 
OpenGL core profile version string: 3.3 (Core Profile) Mesa 12.1.0-devel
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.0 Mesa 12.1.0-devel
OpenGL shading language version string: 1.30
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 12.1.0-devel
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
OpenGL ES profile extensions:

#30 Updated by Joel Palmius almost 2 years ago

Do we think this issue can be solved within the near future? As in, should we block the 1.1.1 release on this?

Atm, I'm inclined to push it to 1.1.2 and add a note about this in the 1.1.1 release notes.

#31 Updated by Rob Baer almost 2 years ago

It has seemed to affect only Ubuntu 16.04 so far. I'm good with pushing it, but it is more important to Aranuvir who lives with it constantly.

Even with the additional information here, I don't see a fix as immediate. Jonas battled through one of these before and it took quite some time for him to find it. That time it was in the Windows side. If it gets fixed fast we can have a fast 1.1.2 and move on to 1.1.3.

Do check out the missing lit shader file #1138 to make sure it's in the repository it needs to be in to make it into the builds. Maybe it is on Tuxfamily and that is intended, but it seems like an environmental item that should be on bitbucket.

#32 Updated by Aranuvir # almost 2 years ago

I've tested all versions of Ubuntu available, from 16.10 down to 14.04.5LTS. All the installations showed the issue. Though testing 14.04.5 wasn't very smart, because of this:

The final point release, 14.04.5, provided the latest Linux kernel and graphics stacks from Ubuntu 16.04 LTS

(Source: https://en.wikipedia.org/wiki/Ubuntu_version_history#Ubuntu_14.04_LTS_.28Trusty_Tahr.29)

Furthermore same issue on Debian. And I'm convinced, that Fedora will show the issue with its next release (F 26), too, when changing their Intel graphics driver...

And unfortunately the litsphere thing doesn't change anything. The 'missing' file is still floating around in my home dir. Focusing on the skin in this threat, in fact the issue affects all other assets like clothing, hair, etc., too.

#33 Updated by Rob Baer almost 2 years ago

@Aranuvir

I believe you when you say that this will get worse. Is it reasonable to make a 1.1.1 release for the unicode issue, and make this a 1.1.2 focal point? or would you vote to hold up 1.1.1 with the likelihood of a near-term solution? I wish I saw such a thing coming.

#34 Updated by Aranuvir # almost 2 years ago

In my opinion the issue should not block the 1.1.1 release. Feel free to move it to 1.1.2 or even 1.2.0.

#35 Updated by Joel Palmius almost 2 years ago

  • Target version changed from MakeHuman 1.1.1 to MakeHuman 1.1.2

There's no way we're fixing this for 1.1.1.

I'm thinking we might be able to find a workaround for 1.1.2 even if we don't fix the underlying problem though.

#36 Updated by Aranuvir # over 1 year ago

  • Related to Bug #1170: Opengl issues on Windows in MakeHuman for Python3 added

#37 Updated by Aranuvir # over 1 year ago

  • Related to Bug #1203: ubuntu 16.04 intel graphics -- shader update and opengl exception added

#38 Updated by Aranuvir # about 1 year ago

  • Status changed from Accepted to Fix exists, needs testing

Also available in: Atom