Generate mesh using custom vertex data

Hi.
I’m new to Metal/iOS and I’m trying to generate mesh using custom vertex data.
In my demo project, the vertex position data is stored in test.bin, and index data is saved in body-index.txt.

I tried to parse vertex position data, and passed the data to vertex buffer, but it seemed that something is wrong while passing data to vertex buffer(The number of vertex does not match).

Can anyone help me with this? Thanks in advance.

I attached my demo project here: BGRenderEngine.zip - Google Drive

btw, Here’s the correct number of vertex and indices and the expectation result like this (implemented using Unity3D)

image

Have you tried with a much smaller, simpler file?

The vertex descriptor that you’re using in init(name:vertexData:indicesData) looks like this:

Screen Shot 2021-10-05 at 8.10.01 am

Is your binary data in this format?

Hi, @caroline ,
Thank you for the reply.

My binary data is not very complated.

  • test.bin: contains vertex position data only.
  • body-index.txt: contains index data only.

But I want to generate normal data using addNormals() at runtime.
So should I just keep position/normal/tangent/bitangent attributes only?

And I don’t have much smaller, simpler file, so I cannot test it with simple data, sorry.

addTangentBasis(forTextureCoordinateAttributeNamed:tangentAttributeNamed:bitangentAttributeNamed:) needs texture coordinates. Do you have those?

  • “Generates surface tangent and bitangent data for the mesh based on its vertex position and texture coordinate data.”

Hi, @caroline

addTangentBasis(forTextureCoordinateAttributeNamed:tangentAttributeNamed:bitangentAttributeNamed:) needs texture coordinates. Do you have those?

The binary file does not contains uv coordinate data.

I’ve tried stripping out all data except position data, and rendering triangles in red.

Assuming that the .bin file really does only contain float3 position, I’ve tried simplifying the vertex shader by taking out all attributes except position just to see if we can render a model. And taking out any uniform data.

And simplifying the fragment shader just to return red.

Cutting down indexData to hold only 256 elements, this is what I get when I render, with lines, the plain data.

Screen Shot 2021-10-05 at 11.47.34 am

It doesn’t help that the GPU debugger is crashing and I can’t examine what is actually going to the GPU.

If I were tackling this problem, I would take it back to first principles, and create a new project with a very simple renderer without all the extra book code in there, and just see if I could render the model without any shading or extra attributes to start with.

1 Like

I created a simpler project, as I had something sitting around.

When I first ran it, I forgot that my project was rendering points and not triangles, and interestingly, I got this result:

Screen Shot 2021-10-05 at 12.59.58 pm

Changing to triangles resulted in a mess.

I don’t know what conclusion to draw from that, but here’s the project:

Vertex.zip (840.3 KB)

Line 138 in Renderer is where you can change the render back to points instead of triangles.

(Also, in that draw call, it should be .uint32 and not .uint16 to match the index buffer. Then you get both sides of the figure, not just a half!)

Edit: Going back to your original question, the number of vertices loaded is correct. The count of rArray is 96234, which, when divided by 3 (because there are three floats in a vertex), comes out to 32078

Edit:

With this project I was able to look at the buffer on the GPU, and I think it’s actually four floats making up a vertex, not three.

Changing the vertex descriptor to a float4 got this:

Screen Shot 2021-10-05 at 2.37.02 pm

Vertex.zip (851.5 KB)

There’s still obviously something weird about the data, but that’s as far as I can take it!

1 Like

Hi, @caroline
Thank you very much for confirming my src code and data!

With this project I was able to look at the buffer on the GPU, and I think it’s actually four floats making up a vertex, not three.
Changing the vertex descriptor to a float4 got this:

It’s weird that changing from float3 to float4 worked somehow.
The app I created using Unity is parsing the binary data into a Vector3 array. and it works well.

I cannot figure out why 4 floats are necessary for one vertex.

I tried to read the demo project you created and investigate the data sent to GPU, and it seems that there’s one vertex data missing.
(The length of float array is 96234, so the number of vertex data is 96234 / 3 = 32078,
so I think the correct last index of vertex data should be 32077, am I wrong?)

I attached the Unity script here, and what I want to do is converting the unity project to iOS native code. (I removed the unnecessary parts) hope it can help.

public class BinaryMeshDataLoader
{
    private int[] indices;

    private TextAsset indexTextAsset;

    private ComputeShader meshGenerateCS;
    private int kernelHandle;

    private Matrix4x4 transformation;

    public void Init(TextAsset textAsset, Matrix4x4 transformation)
    {
        if (indexTextAsset != textAsset)
        {
            indexTextAsset = textAsset;
        }
        this.transformation = transformation;
        if(!meshGenerateCS)
        {
            meshGenerateCS = Resources.Load<ComputeShader>("MeshCompute");
            kernelHandle = meshGenerateCS.FindKernel("CSMain");
        }
    }

    public Mesh LoadData(byte[] modelData)
    {
        if(modelData is null || !indexTextAsset)
        {
            return null;
        }

        if (indices == null || indices.Length == 0)
        {
            indices = ReadIndexFile();
        }

        var newMesh = GenerateMesh(model.data);

        return newMesh;
    }

    private Mesh GenerateMesh(byte[] buffer)
    {
        if(buffer == null || buffer.Length == 0)
        {
            return null;
        }
        Vector3[] vertexArray = new Vector3[buffer.Length / 12];
        ComputeBuffer dataBuf = new ComputeBuffer(buffer.Length / 12, 12);
        dataBuf.SetData(buffer);
        meshGenerateCS.SetBuffer(kernelHandle, "RawDataBuffer", dataBuf);
        meshGenerateCS.SetMatrix("Transformation", transformation);
        meshGenerateCS.SetInt("Size", buffer.Length / 12);

        meshGenerateCS.Dispatch(kernelHandle, buffer.Length / 12, 1, 1);
        dataBuf.GetData(vertexArray);
        Mesh newMesh = new Mesh();
        newMesh.vertices = vertexArray;
        newMesh.triangles = indices;
        newMesh.RecalculateNormals();
        newMesh.RecalculateBounds();
        dataBuf.Dispose();
        return newMesh;
    }

    private int[] ReadIndexFile()
    {
        List<int> indexList = new List<int>();

        var lines = indexTextAsset.text.Split('\n');

        foreach(var line in lines)
        {
            char startLetter = line[0];
            if(startLetter == 'g')
            {
                continue;
            }
            string _line = line.TrimEnd('\r', '\n');
            var indices = GetIndices(_line);
            indexList.AddRange(indices);
        }
        return indexList.ToArray();
    }

    private int[] GetIndices(string line)
    {
        var faceElements = line.Split(' ');
        var index0 = int.Parse(faceElements[1].Split('/')[0]) - 1;
        var index1 = int.Parse(faceElements[2].Split('/')[0]) - 1;
        var index2 = int.Parse(faceElements[3].Split('/')[0]) - 1;
        var index3 = int.Parse(faceElements[4].Split('/')[0]) - 1;

        return new int[] { index0, index1, index2, index0, index2, index3 };
    }
}

And compute shader code here:

#pragma kernel CSMain


RWStructuredBuffer<float3> RawDataBuffer;
float4x4 Transformation;
uint Size;

[numthreads(16,1,1)]
void CSMain (uint3 id : SV_DispatchThreadID)
{
    if(id.x < Size)
    {
        RawDataBuffer[id.x] = mul(Transformation, float4(RawDataBuffer[id.x], 1.0));
    }
}

it seemed that the mesh is generated correctly when I tried to render it in SceneKit, I’m still wondering why the avatar looks weird when rendering it in Metal though.

Thank you very much for the help, @caroline
image

@lanhaide

Sorry it took a while to get back to you :unamused:.

You’re right :slight_smile: - it’s float3. I think I previously had a mismatch between the pipeline vertex descriptor and the mesh vertex descriptor which fogged things.

There’s a difference in stride between Float.stride * 3 and float3.stride. The first is 12 bytes and the second is 16 bytes.

Screen Shot 2021-10-11 at 8.01.43 am

Last vertex number 32077 achieved.

Also, with my render, some of the vertices weren’t rendering because they were outside NDC. When I added +0.2 to the z value in the vertex shader, they showed up (back faces culled).

Screen Shot 2021-10-11 at 8.00.15 am

Aside from changing the z value, I changed the vertex descriptor to use the ones in VertexDescriptor.swift.

Vertex.zip (856.7 KB)

1 Like

Hi, @caroline
Sorry for the late reply.
Thank you for your kind help.
Your code is very useful for me.
I’m very exited to keep on Metal learning. :innocent:

1 Like