How to properly link schematic symbol to graphical one?

1) use list contact view and not cross for all Xref .
2) For graphical elements change the property thumbnail to slave and select "other"

To explain concisely what the issue is: The default library elements are mostly of type "simple".  In order to link different symbols together, you need to change on to be of type "master", then any other symbol of type "slave" can be linked to it.  There are different categories (organ of protection, contactor, etc.) for the master, but the slave just has a "contact type" property which influences how the xrefs are displayed on the master.

You can import the element from the library and then edit it in your project, switching to a different type.  Or better save it to your user collection for later re-use.  Note that you need to import (place) it again after making changes in the user collection, but the properties on already placed symbols will only change after the project is closed and re-opened.

I usually do this for most of my parts, using the "physical view" for the master element. But I also create most elements myself, such as PLC components comprised of different logical groups, all linked to a physical view of the component.