Flexbox Froggy
378 by saaspirant | 58 comments on
New best story on News: Show HN: Unity like game editor running in pure WASM
Show HN: Unity like game editor running in pure WASM
522 by TrevorSundberg | 115 comments on News.
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/nSPMDEz... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/aoJmPKO
522 by TrevorSundberg | 115 comments on News.
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/nSPMDEz... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/aoJmPKO
New best story on Hacker News: Show HN: Unity like game editor running in pure WASM
Show HN: Unity like game editor running in pure WASM
522 by TrevorSundberg | 115 comments on
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/m7SwJWr... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/FgEWs3G
522 by TrevorSundberg | 115 comments on
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/m7SwJWr... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/FgEWs3G
New best story on News: Show HN: Unity like game editor running in pure WASM
Show HN: Unity like game editor running in pure WASM
520 by TrevorSundberg | 115 comments .
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/m7SwJWr... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/FgEWs3G
520 by TrevorSundberg | 115 comments .
In the wake of all the Unity nonsense, just wanted to toss the Raverie engine into this mix :) We’re building off a previous engine that we worked on for DigiPen Institute of Technology called the Zero Engine with a similar component based design architecture to Unity. Our engine had a unique feature called Spaces: separate worlds/levels that you can instantiate and run at the same time, which became super useful for creating UI overlays using only game objects, running multiple simulations, etc. The lighting and rendering engine is scriptable, and the default deferred rendering implementation is based on the Unreal physically based rendering (PBR) approach. The physics engine was built from the ground up to handle both 2D and 3D physics together. The scripting language was also built in house to be a type safe language that binds to C++ objects and facilitates auto-complete (try it in editor!) This particular fork by Raverie builds both the engine and editor to WebAssembly using only clang without Emscripten. We love Emscripten and in fact borrowed a tiny bit of exception code that we’d love to see up-streamed into LLVM, however we wanted to create a pure WASM binary without Emscripten bindings. We also love WASI too though we already had our own in memory virtual file system, hence we don’t use the WASI imports. All WASM imports and exports needed to run the engine are defined here: https://ift.tt/m7SwJWr... The abstraction means that in the future, porting to other platforms that can support a WASM runtime should be trivial. It’s our dream to be able to export a build of your game to any platform, all from inside the browser. Our near term road-map includes getting the sound engine integrated with WebAudio, getting the script debugger working (currently freezes), porting our networking engine to WebRTC and WebSockets, and getting saving/loading from a database instead of browser local storage. Our end goal is to use this engine to create an online Flash-like hub for games that people can share and remix, akin to Scratch or Tinkercad. https://ift.tt/FgEWs3G
New best story on News: Show HN: Paisa – Open-Source Personal Finance Manager
Show HN: Paisa – Open-Source Personal Finance Manager
469 by ananthakumaran | 199 comments .
I have been using plaintext accounting for some time and had a duct-taped together reporting system. Paisa is my latest attempt at making it usable for others. I am interested in knowing what people normally want to understand about their finances PS: Please avoid editing the demo data. Download and run locally if you want to edit.
469 by ananthakumaran | 199 comments .
I have been using plaintext accounting for some time and had a duct-taped together reporting system. Paisa is my latest attempt at making it usable for others. I am interested in knowing what people normally want to understand about their finances PS: Please avoid editing the demo data. Download and run locally if you want to edit.
New best story on Hacker News: Show HN: Paisa – Open-Source Personal Finance Manager
Show HN: Paisa – Open-Source Personal Finance Manager
452 by ananthakumaran | 196 comments on
I have been using plaintext accounting for some time and had a duct-taped together reporting system. Paisa is my latest attempt at making it usable for others. I am interested in knowing what people normally want to understand about their finances PS: Please avoid editing the demo data. Download and run locally if you want to edit.
452 by ananthakumaran | 196 comments on
I have been using plaintext accounting for some time and had a duct-taped together reporting system. Paisa is my latest attempt at making it usable for others. I am interested in knowing what people normally want to understand about their finances PS: Please avoid editing the demo data. Download and run locally if you want to edit.
New best story on News: My uBlock Origin filters to remove distractions
My uBlock Origin filters to remove distractions
415 by mig4ng | 166 comments .
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
415 by mig4ng | 166 comments .
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
New best story on News: My uBlock Origin filters to remove distractions
My uBlock Origin filters to remove distractions
412 by mig4ng | 166 comments on News.
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
412 by mig4ng | 166 comments on News.
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
New best story on Hacker News: My uBlock Origin filters to remove distractions
My uBlock Origin filters to remove distractions
412 by mig4ng | 166 comments on
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
412 by mig4ng | 166 comments on
Repository with my filter lists that block some distractions from sites I want to keep using. I am pretty ruthless removing distractions from my life (e.g. no Instagram, Facebook, TikTok), but some tools I'd like to keep using some parts of it. E.g. Twitter/X, I dislike the feed but I like reading some threads that are shared here or on blog posts. Same for YouTube, I enjoy some videos but I do not want recommendations when I finish the video I was watching. Feel free to suggest more, open issues, pull requests or send me an email :)
New best story on Hacker News: Show HN: HyperDX – open-source dev-friendly Datadog alternative
Show HN: HyperDX – open-source dev-friendly Datadog alternative
443 by mikeshi42 | 107 comments on
Hi HN, Mike and Warren here! We've been building HyperDX (hyperdx.io). HyperDX allows you to easily search and correlate logs, traces, metrics (alpha), and session replays all in one place. For example, if a user reports a bug “this button doesn't work," an engineer can play back what the user was doing in their browser and trace API calls back to the backend logs for that specific request, all from a single view. Github Repo: https://ift.tt/cD9hQ2K Coming from an observability nerd background, with Warren being SRE #1 at his last startup and me previously leading dev experience at LogDNA/Mezmo, we knew there were gaps in the existing tools we were used to using. Our previous stack of tools like Bugsnag, LogRocket, and Cloudwatch required us to switch between different tools, correlate timestamps (UTC? local?), and manually cross-check IDs to piece together what was actually happening. This often made meant small issues required hours of frustration to root cause. Other tools like Datadog or New Relic come with high price tags - when estimating costs for Datadog in the past, we found that our Datadog bill would exceed our AWS bill! Other teams have had to adjust their infrastructure just to appease the Datadog pricing model. To build HyperDX, we've centralized all the telemetry in one place by leveraging OpenTelemetry (a CNCF project for standardizing/collecting telemetry) to pull and correlate logs, metrics, traces, and replays. In-app, we can correlate your logs/traces together in one panel by joining everything automatically via trace ids and session ids, so you can go from log <> trace <> replay in the same panel. To keep costs low, we store everything in Clickhouse (w/ S3 backing) to make it extremely affordable to store large amounts of data (compared to Elasticsearch) while still being able to query it efficiently (compared to services like Cloudwatch or Loki), in large part thanks to Clickhouse's bloom filters + columnar layout. On top of that, we've focused on providing a smooth developer experience (the DX in HyperDX!). This includes features like native parsing of JSON logs, full-text search on any log or trace, 2-click alert creation, and SDKs that help you get started with OpenTelemetry faster than the default OpenTelemetry SDKs. I'm excited to share what we've been working with you all and would love to hear your feedback and opinions! Hosted Demo - https://ift.tt/OuvtZFS Open Source Repo: https://ift.tt/cD9hQ2K Landing Page: https://hyperdx.io
443 by mikeshi42 | 107 comments on
Hi HN, Mike and Warren here! We've been building HyperDX (hyperdx.io). HyperDX allows you to easily search and correlate logs, traces, metrics (alpha), and session replays all in one place. For example, if a user reports a bug “this button doesn't work," an engineer can play back what the user was doing in their browser and trace API calls back to the backend logs for that specific request, all from a single view. Github Repo: https://ift.tt/cD9hQ2K Coming from an observability nerd background, with Warren being SRE #1 at his last startup and me previously leading dev experience at LogDNA/Mezmo, we knew there were gaps in the existing tools we were used to using. Our previous stack of tools like Bugsnag, LogRocket, and Cloudwatch required us to switch between different tools, correlate timestamps (UTC? local?), and manually cross-check IDs to piece together what was actually happening. This often made meant small issues required hours of frustration to root cause. Other tools like Datadog or New Relic come with high price tags - when estimating costs for Datadog in the past, we found that our Datadog bill would exceed our AWS bill! Other teams have had to adjust their infrastructure just to appease the Datadog pricing model. To build HyperDX, we've centralized all the telemetry in one place by leveraging OpenTelemetry (a CNCF project for standardizing/collecting telemetry) to pull and correlate logs, metrics, traces, and replays. In-app, we can correlate your logs/traces together in one panel by joining everything automatically via trace ids and session ids, so you can go from log <> trace <> replay in the same panel. To keep costs low, we store everything in Clickhouse (w/ S3 backing) to make it extremely affordable to store large amounts of data (compared to Elasticsearch) while still being able to query it efficiently (compared to services like Cloudwatch or Loki), in large part thanks to Clickhouse's bloom filters + columnar layout. On top of that, we've focused on providing a smooth developer experience (the DX in HyperDX!). This includes features like native parsing of JSON logs, full-text search on any log or trace, 2-click alert creation, and SDKs that help you get started with OpenTelemetry faster than the default OpenTelemetry SDKs. I'm excited to share what we've been working with you all and would love to hear your feedback and opinions! Hosted Demo - https://ift.tt/OuvtZFS Open Source Repo: https://ift.tt/cD9hQ2K Landing Page: https://hyperdx.io
New best story on News: Fine-tune your own Llama 2 to replace GPT-3.5/4
Fine-tune your own Llama 2 to replace GPT-3.5/4
463 by kcorbitt | 123 comments on News.
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/7YAd1zh ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/pDjRL1Z... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/D7QiMOJ with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/lYX56xi ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
463 by kcorbitt | 123 comments on News.
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/7YAd1zh ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/pDjRL1Z... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/D7QiMOJ with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/lYX56xi ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
New best story on Hacker News: Fine-tune your own Llama 2 to replace GPT-3.5/4
Fine-tune your own Llama 2 to replace GPT-3.5/4
461 by kcorbitt | 122 comments on
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/vr8I1x2 ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/3ZpXIdb... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/NGTb041 with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/h8FjtnE ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
461 by kcorbitt | 122 comments on
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/vr8I1x2 ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/3ZpXIdb... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/NGTb041 with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/h8FjtnE ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
New best story on News: Fine-tune your own Llama 2 to replace GPT-3.5/4
Fine-tune your own Llama 2 to replace GPT-3.5/4
446 by kcorbitt | 118 comments .
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/vr8I1x2 ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/3ZpXIdb... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/NGTb041 with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/h8FjtnE ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
446 by kcorbitt | 118 comments .
There has been a lot of interest on HN in fine-tuning open-source LLMs recently (eg. Anyscale's post at https://ift.tt/vr8I1x2 ). I've been playing around with fine-tuning models for a couple of years, and wanted to share some insights and practical code. I’ve condensed what I’ve learned into a small set of notebooks at https://ift.tt/3ZpXIdb... , covering labeling data, fine-tuning, running efficient inference, and evaluating costs/performance. The 7B model we train here matches GPT-4’s labels 95% of the time on the test set, and for the 5% of cases where they disagree it’s often because the correct answer is genuinely ambiguous. What is fine-tuning? You can think of it as a more-powerful form of prompting, where instead of writing your instructions in text you actually encode them in the weights of the model itself. You do this by training an existing model on example input/output pairs that demonstrate the task you want your fine-tuned model to learn. Fine-tuning can work with as few as 50 examples but I usually try to get 1000+ if possible. Prompting still has some big advantages over fine-tuning. It's way easier/faster to iterate on your instructions than label data and re-train a model. And operationally it's easier to deploy one big model and just adjust its behavior as necessary vs deploying many small fine-tuned models that will likely each get lower utilization. Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://ift.tt/NGTb041 with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. Disclaimer: My brother David and I are working on an open-source product called OpenPipe ( https://ift.tt/h8FjtnE ) to help engineers adopt fine-tuning as simply as possible. But none of the information above depends on our startup. The current post is just about sharing information that we’ve learned about fine-tuning. I hope it’s useful!
New best story on News: Show HN: Host a Website in the URL
Show HN: Host a Website in the URL
442 by acidx | 144 comments on News.
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
442 by acidx | 144 comments on News.
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
New best story on Hacker News: Show HN: Host a Website in the URL
Show HN: Host a Website in the URL
441 by acidx | 143 comments on
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
441 by acidx | 143 comments on
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
New best story on News: Show HN: Host a Website in the URL
Show HN: Host a Website in the URL
439 by acidx | 143 comments .
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
439 by acidx | 143 comments .
I wrote this silly thing a couple of weeks ago. It's absolutely useless but it's a fun tech demo for my web server library. Enjoy!
New best story on News: Show HN: Puck – Open-source visual editor for React
Show HN: Puck – Open-source visual editor for React
396 by chrisvxd | 86 comments on News.
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
396 by chrisvxd | 86 comments on News.
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
New best story on Hacker News: Show HN: Puck – Open-source visual editor for React
Show HN: Puck – Open-source visual editor for React
396 by chrisvxd | 86 comments on
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
396 by chrisvxd | 86 comments on
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
New best story on News: Show HN: Puck – Open-source visual editor for React
Show HN: Puck – Open-source visual editor for React
395 by chrisvxd | 86 comments .
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
395 by chrisvxd | 86 comments .
Hey hackers, OP here! I've been dipping in and out of this problem space for the last few years with many of my clients. Puck sits somewhere between an old-school WYSIWYG-powered CMS and headless one, allowing content teams to author content using real React components. Traditional CMS solutions were flexible but often resulted in page that completely broke the brand guidelines. Headless CMS solutions are a fantastic way of controlling brand by restricting UI changes to developers, but makes layout changes restrictive and slow as developers often need to get involved. Puck provides a visual editor for React that can sit on top of your existing headless CMS (or act as standalone). We've been dog-fooding it on a few pages at https://measured.co and on https://wellpaid.io . So far, so good The API is built for React, which allows FE devs to quickly integrate their existing component and add some form fields for author input, or connect it to a headless CMS of choice. It's open-source under MIT, and pairs nicely with Next.js (check out the demo application). Next in the pipeline: support for multi-column layouts, richer demos, new plugins. Looking forward to hearing your comments!
Subscribe to:
Posts (Atom)