I just installed 13.6.1 on an M1 MacBook Pro and am now facing the boot issue, so I'm guessing it's not fixed :). To make matters worse, the specific USB port required for the DFU revive fix is broken as well, which was never an issue as the other two worked... oof.
We currently use the kafka connect bigquery connector, along with debezium, to "stream" both the "changelog"/"transaction log" and to "mirror" our rdbms instances into bigquery. While this works, it's been a fair amount of effort to iron out issues over time. We also have had to work around bigquery limits including issues exceeding concurrent queries (switched to batch mode, which has it's own issues) and frequency of writes (we've had to throttle to flushing every minute, which is good enough, but did have a use case for faster updates). Also have issues related to partitioning and clustering, and more...
So seeing this to potentially replace the kafka connect bigquery connector looked appealing. However, according to the docs and listed limitations (https://cloud.google.com/datastream/docs/sources-postgresql) it does not handle schema changes well nor postgres array types. Not that any of these tools handle this well, but given the open source bigquery connector, we've been able to work around this with customizations to the code. Hopefully they'll continue to iterate on the product and I'll be keeping an eye out.
Yeah the Debezium connectors have some issues that really get in the way. I'm less familiar with BQ but some other DBs, the data typing is really, really basic
(case when string then varchar(4000/max))
and similar. It looks like a relatively easy thing to incrementally improve.
Hey, Gunnar here from the Debezium team. I would love to learn more about those issues you encountered with Debezium. Perhaps it's something we could improve. So if you could share your feedback either here, on our mailing list (https://groups.google.com/g/debezium), that would be awesome. Thanks!
For us, debezium has been working rather well in recent memory (there were a couple PRs I submitted that I'm proud of, even if tiny!). Most of the issues are on the kafka connector side, whether it's the bigquery sink, jdbc sink, and s3 sink.
A couple things that do pop to mind as it relates to debezium include better capabilities around backup + restores and disaster recovery, and any further hints around schema changes. Admittedly I haven't looked at these areas for 6+ months so they may be improved.
Well I'm trying to find it now! Maybe it has been updated. It was essentially a case statement on the destinations which went for some wide data types but, looking now, I see where it is looking at the schemas and getting more precision.
I'd really like to try this out on a postgres replica, however these instances don't appear in the web console at least when creating a replica from an existing x86 primary. Anyone have insight here?
NEXT Trucking | El Segundo, CA (Headquarters) & Irvine / Orange County, CA | Onsite | Full-time
NEXT Trucking builds software for modernizing & optimizing drayage, the process of moving shipping containers from ports to local warehouses, with a focus on 'Drivers First'. This video, featuring our CEO Lidia Yan, describes the business and market opportunity much better than I can: https://www.youtube.com/watch?v=VHHI3vaqCS4
We're hiring software developers, platform/SREs, data engineers & scientists, UI/UX designers, product managers, project managers, and more: https://www.nexttrucking.com/careers/
Highlights of our tech stack include Java + Spring Boot, React + React Native, Storybook, GraphQL, Node, NestJS, PostgreSQL, OpenTracing (Jaeger), Kafka, and deployed to AWS + Kubernetes via Terraform. We write lots of tests (perhaps even too many!), are continuously learning & improving our processes, open source friendly, and have a warm, inclusive, & welcoming culture.
We're also not picky about candidates having used the same technologies we use today, having hired developers with heavy backgrounds in C#, Angularjs, Python, Perl, etc.
Who am I? I'm Kevin, an IC working primarily from our Irvine location on platform and infrastructure bits, and would love to have more folks from Orange County seize the opportunity to work for a well funded series-C startup, without having to deal with LA traffic : ) I'm more than happy to chat and answer questions, so please feel free to shoot me an email: kevinp@nexttrucking.com .
NEXT Trucking | El Segundo, CA (Headquarters) & Irvine / Orange County, CA | Onsite | Full-time
NEXT Trucking builds software for modernizing & optimizing drayage, the process of moving shipping containers from ports to local warehouses, with a focus on 'Drivers First'. This video, featuring our CEO Lidia Yan, describes the business and market opportunity much better than I can: https://www.youtube.com/watch?v=VHHI3vaqCS4
We're hiring software developers, platform/SREs, data engineers & scientists, UI/UX designers, product managers, project managers, and more: https://www.nexttrucking.com/careers/
Highlights of our tech stack include Java + Spring Boot, React + React Native, Storybook, GraphQL, Node, NestJS, PostgreSQL, OpenTracing (Jaeger), Kafka, and deployed to AWS + Kubernetes via Terraform. We write lots of tests (perhaps even too many!), are continuously learning & improving our processes, open source friendly, and have a warm, inclusive, & welcoming culture.
We're also not picky about candidates having used the same technologies we use today, having hired developers with heavy backgrounds in C#, Angularjs, Python, Perl, etc.
Who am I? I'm Kevin, an IC working primarily from our Irvine location on platform and infrastructure bits, and would love to have more folks from Orange County seize the opportunity to work for a well funded series-C startup, without having to deal with LA traffic : ) I'm more than happy to chat and answer questions, so please feel free to shoot me an email: kevinp@nexttrucking.com .
NEXT Trucking | El Segundo, CA (Headquarters) & Irvine / Orange County, CA | Onsite | Full-time
NEXT Trucking builds software for modernizing & optimizing drayage, the process of moving shipping containers from ports to local warehouses, with a focus on 'Drivers First'. This video, featuring our CEO Lidia Yan, describes the business and market opportunity much better than I can: https://www.youtube.com/watch?v=VHHI3vaqCS4
Highlights of our tech stack include Java + Spring Boot, React + React Native, Storybook, GraphQL, Node, NestJS, PostgreSQL, OpenTracing (Jaeger), Kafka, and deployed to AWS + Kubernetes via Terraform. We write lots of tests (perhaps even too many!), are continuously learning & improving our processes, open source friendly, and have a warm, inclusive, & welcoming culture.
We're also not picky about candidates having used the same technologies we use today, having hired developers with heavy backgrounds in C#, Angularjs, Python, Perl, etc.
Who am I? I'm Kevin, an IC working primarily from our Irvine location on platform and infrastructure bits, and would love to have more folks from Orange County seize the opportunity to work for a well funded series-C startup, without having to deal with LA traffic : ) I'm more than happy to chat and answer questions, so please feel free to shoot me an email: kevinp@nexttrucking.com .
NEXT Trucking | El Segundo, CA (Headquarters) & Irvine / Orange County, CA | Onsite | Full-time
NEXT Trucking builds software for modernizing & optimizing drayage, the process of moving shipping containers from ports to local warehouses, with a focus on 'Drivers First'. This video, featuring our CEO Lidia Yan, describes the business and market opportunity much better than I can: https://www.youtube.com/watch?v=VHHI3vaqCS4
Highlights of our tech stack include Java + Spring Boot, React + React Native, Storybook, GraphQL, Node, NestJS, PostgreSQL, OpenTracing (Jaeger), Kafka, and deployed to AWS + Kubernetes via Terraform. We write lots of tests (perhaps even too many!), are continuously learning & improving our processes, open source friendly, and have a warm, inclusive, & welcoming culture.
We're also not picky about candidates having used the same technologies we use today, having hired developers with heavy backgrounds in C#, AngularJS, Python, Perl, etc.
Who am I? I'm Kevin, an IC working primarily from our Irvine location on platform and infrastructure bits, and would love to have more folks from Orange County seize the opportunity to work for a well funded series-C startup, without having to deal with LA traffic : ) I'm more than happy to chat and answer questions, so please feel free to shoot me an email: kevinp@nexttrucking.com .
While a bit of a drive from LA, if you haven't made a trip out to Oak Glen I'd suggest a look at the various apple orchards out there: https://www.oakglen.net/ .
This place for example lists 25 or so varieties http://losriosrancho.com/produce/ and all the orchards I've visited over the years provide samples of most, if not all, of their stock.
While it feels a bit hacky and unclean, you may want to try using IKVM (http://www.ikvm.net/) to translate and import the Java client in to your .NET project.
Given the difficulty in building a client period (distributed systems, race conditions, etc), being able to rely on the widely adopted & supported official client is quite attractive.
In my test cases the performance is on par running natively on the JVM, except when compression is enabled.
Another option is using the REST proxy and accepting the trade-offs that imposes.
I've faced similar issues in the .NET world. There is a built in method, `GC.KeepAlive()`, that does nothing other than keep the reference alive to the eyes of the garbage collector.
It completely defeats the whole point of a garbage collector if you have to resort to hacks like this. I understand why it is done. A lot easier to use the work around then to fix the root problem.
Unfortunatelly these types of bugs have a nasty way of showing up in other places where it can be even harder to debug. Long term these bugs need to be fixed otherwise your code will just be a ticking bomb. The worst part is when they combine with other bugs and they become even harder to debug.
I wouldn't say "completely defeats". The hacks are usually there only when you need to manage the lifetime of something that's not managed by the garbage collector. I've seen it used for mutexes, for example, to keep a mutex alive and locked.
You could say that the problem is that people abuse the garbage collector to manage something it shouldn't manage, or you could say that the problem is that garbage collectors are bad at managing anything that isn't memory. (I would lean towards the latter.)
nope, when you have handles on resources outside managed memory, the GC is out and you need to have an acquire/release mechanism. This resource will probably have a bit of state visible on the managed side, so there will be some pinned memory.
