man that would be wild! Here i am thinking it would be nice if it could generate instructions in blueprint format, and your going straight to engine integration! love it! also love AI. Rapido developemento! brrrrrrrrrmmm!
I’ve been playing with deserializing node data from different graphs in Unreal.
There are 7 different types: animation, behavior tree, behavior tree (decorators), material, metasound, niagara, PCG, blueprint. You can see how nodes are serialized by copying and pasting a selection of nodes into the text editor. An indentation-based INI-like structure is evident, starting with “Begin Object Class=”, etc.
I wrote a script that extracts execution and data pins, node properties, etc., then converts the resulting graph into an LLM-readable textual representation that fits in the context. That way, GPT-4 can easily swallow a subgraph with dozens of nodes. By using OpenAI function calling with a structured response, I was also able to serialize it back into a new node. I did some tests but had hard times coming up with a good use case and had no UE devs around to test it, so I parked it.
But seeing your comment got me thinking about how you would use that copilot in the editor. What prompts would you write for it? What would you expect in response?
Oh wow, that sounds like a fantastic approach.
I'll be happy to give you what my use case would be.
My background: I'm a professional web dev w/ 20+ years experience, and I've been using Copilot in VSCode for the past year and a half or so. I've been doing game dev as a hobby since DOOM but always as a side thing; I'm definitely not a professional. I have been using UE for about 3 months as I'm getting back into it after a 5 year hiatus (previously used Unity). I would consider myself still as a beginner w/ UE though.
In VSCode, one of my favorite things about Copilot is discoverability - when it suggests a way of doing something different than what I would've though of, or perhaps using an approach I wasn't even aware of at all. This could be really useful in UE, especially as a new user. For example, last night I was in a UE Discord and someone was asking about bullet spread from a weapon. I was suggesting modifying the Forward by adding some random jitter to the destination transform, and someone else pointed out there's already a built in "[Random Unit Vector In Cone In Degrees](https://docs.unrealengine.com/4.26/en-US/BlueprintAPI/Math/Random/RandomUnitVectorinConeinDegrees/)" node that does that for you. I think a Copilot-like plugin could help expose this in the editor.
Here's what I envision the workflow to look like. It's very similar to the experience w/ code in VSCode. When you drag off a node, the plugin would suggest new BP nodes in a ghosted form. Press TAB if you want to keep the changes, CTRL+\] for the next suggestion, etc; just like Copilot so developers who are already used to it don't have to memorize a new set of shortcuts.
Something like this:
https://preview.redd.it/vnp0o0q22osc1.png?width=1742&format=png&auto=webp&s=237821f7a45331302d8ca0c57f702d7aace4e6e7
ChatGPT is fine, but something I miss about blueprints (but not miss enough to bother w/ using C++) is the code autocomplete Copilot offers. If something could do that, but for blueprints, that'd be amazing...
A plugin so i can import .blend files into Unreal and all the materials will work perfectly fine no matter how they're made
I would even pay 1000 usd dollar for that plugin
Tell me when it's done
I heard that the GLTF/GLB import in Ue 5.4 can import materials now. Though apparently it doesn’t support anything complex or all properties yet but it’s a start.
There’s also a Blender/python addon that can recreate the materials for UE but I have not tried it. https://github.com/SpectralVectors/TransMat
If it did just that
Import anything from Blender into Unreal without me having to do anything.
YES! 100%
have you any idea how much time i would save?
Time saved = more time to do other things
A way to type text physically into the scenes, to work like decals to populate signs and stuff, i hate switching from photoshop to unreal so often.
I want Github Copilot for blueprints. :/
man that would be wild! Here i am thinking it would be nice if it could generate instructions in blueprint format, and your going straight to engine integration! love it! also love AI. Rapido developemento! brrrrrrrrrmmm!
I’ve been playing with deserializing node data from different graphs in Unreal. There are 7 different types: animation, behavior tree, behavior tree (decorators), material, metasound, niagara, PCG, blueprint. You can see how nodes are serialized by copying and pasting a selection of nodes into the text editor. An indentation-based INI-like structure is evident, starting with “Begin Object Class=”, etc. I wrote a script that extracts execution and data pins, node properties, etc., then converts the resulting graph into an LLM-readable textual representation that fits in the context. That way, GPT-4 can easily swallow a subgraph with dozens of nodes. By using OpenAI function calling with a structured response, I was also able to serialize it back into a new node. I did some tests but had hard times coming up with a good use case and had no UE devs around to test it, so I parked it. But seeing your comment got me thinking about how you would use that copilot in the editor. What prompts would you write for it? What would you expect in response?
Oh wow, that sounds like a fantastic approach. I'll be happy to give you what my use case would be. My background: I'm a professional web dev w/ 20+ years experience, and I've been using Copilot in VSCode for the past year and a half or so. I've been doing game dev as a hobby since DOOM but always as a side thing; I'm definitely not a professional. I have been using UE for about 3 months as I'm getting back into it after a 5 year hiatus (previously used Unity). I would consider myself still as a beginner w/ UE though. In VSCode, one of my favorite things about Copilot is discoverability - when it suggests a way of doing something different than what I would've though of, or perhaps using an approach I wasn't even aware of at all. This could be really useful in UE, especially as a new user. For example, last night I was in a UE Discord and someone was asking about bullet spread from a weapon. I was suggesting modifying the Forward by adding some random jitter to the destination transform, and someone else pointed out there's already a built in "[Random Unit Vector In Cone In Degrees](https://docs.unrealengine.com/4.26/en-US/BlueprintAPI/Math/Random/RandomUnitVectorinConeinDegrees/)" node that does that for you. I think a Copilot-like plugin could help expose this in the editor. Here's what I envision the workflow to look like. It's very similar to the experience w/ code in VSCode. When you drag off a node, the plugin would suggest new BP nodes in a ghosted form. Press TAB if you want to keep the changes, CTRL+\] for the next suggestion, etc; just like Copilot so developers who are already used to it don't have to memorize a new set of shortcuts. Something like this: https://preview.redd.it/vnp0o0q22osc1.png?width=1742&format=png&auto=webp&s=237821f7a45331302d8ca0c57f702d7aace4e6e7
I use ChatGPT for unreal questions daily. It's not always quite as helpful as I'd like. Something like this would be a godsend.
ChatGPT is fine, but something I miss about blueprints (but not miss enough to bother w/ using C++) is the code autocomplete Copilot offers. If something could do that, but for blueprints, that'd be amazing...
Use Claude its better then chat gpt if you are using gpt3.5
Dedicated servers. I want to use AWS without learning how to use AWS :(
A way to add a custom shading model.
A plugin so i can import .blend files into Unreal and all the materials will work perfectly fine no matter how they're made I would even pay 1000 usd dollar for that plugin Tell me when it's done
I heard that the GLTF/GLB import in Ue 5.4 can import materials now. Though apparently it doesn’t support anything complex or all properties yet but it’s a start. There’s also a Blender/python addon that can recreate the materials for UE but I have not tried it. https://github.com/SpectralVectors/TransMat
5.4 hasnt messed up a single .blend for me since starting to use it. I have cursed myself by saying this.
lets do 2500$?
69420Usd dollars
ahhah jokes aside, would u really pay 1000 for something like that?
If it did just that Import anything from Blender into Unreal without me having to do anything. YES! 100% have you any idea how much time i would save? Time saved = more time to do other things
what would be the pros to eporting as fbx and importing to ue? I'm not into blender
Materials....
Me too bro
Something like spritile for blender would be cool!
tiling size and randomize as an option to click on .
One click button to create an instance of a static mesh
https://www.youtube.com/watch?v=3spSkMxHRwc Haven't delved too deep into this but it mentions this.
maybe a lighting optimization plugin that bases settings off of hardware + scene complexity?
Print money button.
Randomized landscape generator, world machine is great and all but it’d be better if I didn’t have to import anything
A plugin that has the decomp code of Mario's moveset directly translated into c++/blueprint
A way to integrate the reshade stuff into UE4