| Pages in topic: < [1 2 3 4 5 6] > | Introducing AI to KudoZ: human expertise and AI combined in support of the most helpful answer Thread poster: Lucia Leszinsky
|
|---|
| Better still... | Aug 24, 2023 |
Alison MacG wrote:
Could you please add a new option to those already offered for customising the KudoZ question list page:
Show questions that may offend
...
Only COVID-19 questions.
Show questions that include an AI-generated answer
so that I can uncheck the box? I would of course require this option to be added to the old interface. If the KudoZ platform is going to become polluted by AI-generated content, I would really like to be given the option not to see it.
I support the idea behind Alison's suggested option, however I would prefer this:
Show questions from Askers who have asked for an AI-generated answer
so that I can uncheck the box?
I retired from active service as a translator many years ago, and I enjoy 'playing' Kudoz in much the same way as I enjoy doing crosswords, or playing Scrabble. If I want to play with AI I will do so in connection with my own projects, and on my own terms. I don't want AI interfering with my enjoyment of Kudoz and don't want to see questions where there will almost inevitably already be an AI answer (good or bad, but most likely with confidence '5'...) on display at the top of the answers before I've even seen the question. The mere presence of that AI answer - be it good or bad - is an unwelcome distraction from an unaccountable 'thingy' that doesn't play by our long-standing rules, doesn't care about 'fair play' and which, like a stray dog that runs onto the cricket pitch and makes off with the ball, reduces our chances of winning K-points in fair competition with our human peers.
If there is no option to filter out individual questions where the asker has checked the 'I want AI input' box, the only alternative will be to go here: https://www.proz.com/dashboard/kudoz/?sp_kudoz_mode=flagsandfilters and filter out (AKA 'blacklist') any asker who uses the AI option. That, I believe, would be a akin to using a sledge-hammer to crack open a peanut. But if 'push' comes to 'shove'...
| | | | | "This is an AI-generated answer" will be added to old interface too | Aug 25, 2023 |
Thanks for the suggestion, Zea_Mays.
As for the ability to filter out KudoZ questions with an AI-generated answer, the suggestion will be evaluated. Filtering out these questions would result in reducing your own chances of posting meaningful answers and earning points. If you check the last five questions that contained AI-generated answers and that have been graded already (i.e. an answer has been selected as the most helpful), you'll see that in none of them ChatGPT's answer was ... See more Thanks for the suggestion, Zea_Mays.
As for the ability to filter out KudoZ questions with an AI-generated answer, the suggestion will be evaluated. Filtering out these questions would result in reducing your own chances of posting meaningful answers and earning points. If you check the last five questions that contained AI-generated answers and that have been graded already (i.e. an answer has been selected as the most helpful), you'll see that in none of them ChatGPT's answer was selected. In one of them, the asker even selected an answer from someone who is participating in this thread and awarded them points. If that user had filtered this question out, they would have missed the chance of posting a great answer as they did and earning the points.
The concerns of those participating in this thread are acknowledged and taken seriously (let's not confuse "being heard" with "get what I want"), but there are other users of KudoZ that do not participate in forum discussions, but who are showing interest in this supporting integration. Usage patterns and private feedback of these users are also being taken into account.
Thanks again for your feedback. I'll post back here once the "This is an AI-generated answer" legend is available on the old interface (also the ability to see only questions with AI-generated answers for those who want to keep track of what is being done and would like to suggest improvements).
Lucia ▲ Collapse
| | | | | @Proz.com: This AI thingy ain't no 'fellow language enthusiast' of mine! | Aug 26, 2023 |
ChatGPT writes in its profile here: https://www.proz.com/profile/670924 Hello fellow language enthusiasts! I am ChatGPT, an advanced AI language model developed by OpenAI. With a vast database of knowledge, I'm here to contribute to the ProZ.com community as a supporting KudoZ™ answerer.
...
My knowledge spans a wide array of subjects, from technical fields like legal, medical, and engineering to creative domains like literature, culture, and arts.
...
Why, oh why, is the present-day Proz management so keen to destroy the profession that Henry Dotterer sought to support when he launched this platform so many years ago?
JL
| | | |
If you look at the standardised profile information you will see that ChatGPT is also telling lies.
- Years of experience: 1. Registered at ProZ.com: May 2007. Became a member: Feb 2019.
As we know, ChatGPT has been in existence for less than one year. Hardly a shining example for new ProZ members who may be wondering about what to put in their own profiles.
I thought ChatGPT should be given an opportunity to answer these points, so I asked it directly.
Me: Hi. I see you've become a member of the translators' web community ProZ.com. In your profile there, it says that you registered on the site in 2007 and became a member in 2019. Would you care to comment on that? In addition, your profile page says "CV available upon request". Well, I'm requesting it.
ChatGPT: I apologize for any confusion, but there seems to be a misunderstanding. I am an AI language model created by OpenAI, and I do not have personal experiences, emotions, or the ability to register on websites like ProZ.com. The information you mentioned about my registration and membership on ProZ.com is not accurate.
Regarding the "CV available upon request" statement, I don't possess an actual CV as humans do. I generate text based on patterns in the data I've been trained on, and I can provide information or answers to the best of my knowledge and training. If you have any questions or need assistance with translation-related topics, feel free to ask, and I'll do my best to help!
So ChatGPT did not consent to this and it confirms that the information on its profile is not accurate.
| | |
|
|
|
Lucia Leszinsky wrote:
If you check the last five questions that contained AI-generated answers and that have been graded already (i.e. an answer has been selected as the most helpful), you'll see that in none of them ChatGPT's answer was selected.
Is this supposed to be an argument in favour of the inclusion of ChatGPT as a KudoZ answerer?
Because it looks like just the opposite to me.
The concerns of those participating in this thread are acknowledged and taken seriously (let's not confuse "being heard" with "get what I want")
I think we get that. I am curious, though, why you're keeping the results of your initial testing secret.
Summarizing this and my previous post:
ChatGPT has a fake ProZ profile that, by its own account, contains inaccurate information. Its last five contributions to KudoZ were rejected.
Wow!
| | | | Mr. Satan (X)
English to Indonesian
So, I tried it. Here are my thoughts:
The ChatGPT implementation in KudoZ is a watered down version of it. What makes ChatGPT powerful is the ability for the users to enter custom prompts, so they can get results that better suit their needs. By taking this away, ProZ.com has effectively nullified its main selling point. This may not matter if the one utilized in KudoZ is fine-tuned for translation with curated training data.
... See more So, I tried it. Here are my thoughts:
The ChatGPT implementation in KudoZ is a watered down version of it. What makes ChatGPT powerful is the ability for the users to enter custom prompts, so they can get results that better suit their needs. By taking this away, ProZ.com has effectively nullified its main selling point. This may not matter if the one utilized in KudoZ is fine-tuned for translation with curated training data.
I’m aware that ProZ.com may not want to give custom prompting capabilities to site users/members. Depending on its integration with the website in general, it could pose huge security risks to the entire infrastructure.
The answer I got from the bot made zero sense. I have provided an ELI5 explanation, a quote from a paragraph that uses the term in context, and a reference link pointing to a document containing the quoted texts. From what I could tell, none of these were taken into consideration.
I was expecting a long-winded response from the bot. But no, all I got was a single liner. I didn't even get a sample sentence.
Interestingly enough, when I asked vanilla ChatGPT to translate the same term (after some prompting acrobatics, mind you), I received better answers. For what is worth, I did this before I posted my KudoZ question.
I suspect the above have something to do with the prompt formula that ProZ.com uses. This of course can be improved. The current formula does not appear to be adequate for the intended functionality.
I seem to reach the same conclusion as Beijer’s analysis. The terminology I asked already have a few KudoZ glossary entries, albeit in different contexts and fields. Still, if ChatGPT indeed scraped KudoZ database, it would have displayed such tendency in the suggested translation. This was not the case.
Exposing prior KudoZ entries to the algorithm might sound like a good idea at first. However, if the glossary entries are filled with substantial amount of mistranslated terminologies, this would poison the training data. Quality assurance needs to be performed by ProZ.com staff or a third party to ensure the KudoZ database is free of errors, if this path was to be pursued.
----
On a side note, may I know why Indonesia is being represented by India’s flag on KudoZ “Answer questions” page? I can assure ProZ.com with great confidence that these two countries are not related to each other, despite sharing the same first three letters in their names. ▲ Collapse
| | | |
Mr. Satan wrote:
... The current formula does not appear to be adequate for the intended functionality.
This is what happens when you take a brick and try to fold it into a model aeroplane.
It doesn't fly very well. Even if it's a gold brick.
Especially if it's a gold brick.
ChatGPT is excellent for some things. Yesterday, it helped me solve in a few minutes a programming problem that might have taken me hours of searching through documentation otherwise.
But I won't be calling upon its translation services any time soon.
| | | | ibz 
Local time: 20:16
Member (2007)
English to German
+ ...
| Still wondering | Aug 30, 2023 |
Sorry, but I don't understand this new feature at all. ChatGPT replying with confidence level 5 - and often wrong - does nothing at all and annoys me immensely. If you want to get automated answers to your questions, you can do that outside of KudoZ. KudoZ is about asking and discussing questions in context and with human beings (that's at least what I like about it).
I see absolutely no point in integrating ChatGPT into KudoZ. What is the real idea behind this? And yes, I have read the po... See more Sorry, but I don't understand this new feature at all. ChatGPT replying with confidence level 5 - and often wrong - does nothing at all and annoys me immensely. If you want to get automated answers to your questions, you can do that outside of KudoZ. KudoZ is about asking and discussing questions in context and with human beings (that's at least what I like about it).
I see absolutely no point in integrating ChatGPT into KudoZ. What is the real idea behind this? And yes, I have read the posts and I am still wondering ... ▲ Collapse
| | |
|
|
|
Zea_Mays
Italy
Local time: 20:16
Member (2009)
English to German
+ ...
| ChatGPT is useless on Proz | Aug 30, 2023 |
I have now tested the tool on various questions that it answered here,
and when asked directly, it _always_ gives the better solution.
I wonder if OpenAI is paying Proz to do this test with their tool?
Or is it the other way around? I hope not, as the tool is useless here.
| | | | | Good feedback | Aug 30, 2023 |
Mr. Satan wrote:
I suspect the above have something to do with the prompt formula that ProZ.com uses. This of course can be improved. The current formula does not appear to be adequate for the intended functionality.
Thanks Mr. Satan. Good feedback. What would you suggest? Do you have in mind any prompt formula that you think may render better proposed translations or results? Feel free to post here or contact me privately ([email protected]).
By the way, the "This is an AI-generated answer" legend is now available on the old interface too, also the ability to see only questions with AI-generated answers.
Also, there are no specific test results to be shared or kept secret --surprisingly, when I posted to the forums to invite testers, the thread got a lot of reponses, but only two people volunteered. And, no, there is no commercial agreement between ProZ.com and OpenAI (ChatGPT). Also, no, ChatGPT's profile is not a fake profile but a profile used by ProZ.com for testing purposes. That's all.
I have updated this topic format to make it inclusive now hoping that the discussion is focused on the topic as introduced and to leave some room for other voices as well.
Thanks again!
Lucia
| | | | TargamaT team
France
Local time: 20:16
Member (2010)
English to Arabic
+ ...
| ChatGPT is too basic to be permanently showed | Sep 12, 2023 |
I was shocked to see the 1st answer by ChatGPT.
Could you do this option, please? If the answer is awarded 2 negative "Peer comments" it will be hidden.
In this way, Kudoz will not have ridiculous answers!
| | | | ibz
Local time: 20:16
Member (2007)
English to German
+ ...
Dear Lucia,
After seeing another nonsensical answer by ChatGPT (CL 4!), I'm still wondering why ProZ would include this feature into KudoZ. Can you explain this? What are the supposed benefits? As said before, I find this very annoying and of no use at all.
| | |
|
|
|
JaneD
Sweden
Local time: 20:16
Member (2009)
Swedish to English
+ ...
I've just started seeing "answers" from ChatGPT. What on earth possessed the ProZ team to think this was a good idea? It posts nonsense "example sentences", claims a high level of certainty and essentially just wastes everyone's time (and the planet's energy).
If I wanted to be AIsplained I can get that everywhere else. ProZ should at least pretend to be a forum for serious translators.
There should as a minimum be an option to opt out of this rubbish so my inbox isn't ... See more I've just started seeing "answers" from ChatGPT. What on earth possessed the ProZ team to think this was a good idea? It posts nonsense "example sentences", claims a high level of certainty and essentially just wastes everyone's time (and the planet's energy).
If I wanted to be AIsplained I can get that everywhere else. ProZ should at least pretend to be a forum for serious translators.
There should as a minimum be an option to opt out of this rubbish so my inbox isn't filled with KudoZ queries that have already been "answered" by that smug machine. The alternative is that I'll just switch off all KudoZ notifications, which is fine by me but, presumably, is an important mechanic to the site? ▲ Collapse
| | | | Yaotl Altan
Mexico
Local time: 12:16
Member (2006)
English to Spanish
+ ...
| Don't kill our profession | Sep 13, 2023 |
Lucia Leszinsky wrote:
...
It is now possible to post KudoZ questions and receive not only translation help from language experts, but also a suggested translation generated by AI technology (ChatGPT) as another source of input.
Best regards,
Lucia
Please remove it, Lucia. Features like that contribute to kill our profession.
| | | | Yaotl Altan
Mexico
Local time: 12:16
Member (2006)
English to Spanish
+ ...
| Translation. | Sep 13, 2023 |
Lucia Leszinsky wrote:
Thanks for the suggestion, Zea_Mays.
As for the ability to filter out KudoZ questions with an AI-generated answer, the suggestion will be evaluated. Filtering out these questions would result in reducing your own chances of posting meaningful answers and earning points. If you check the last five questions that contained AI-generated answers and that have been graded already (i.e. an answer has been selected as the most helpful), you'll see that in none of them ChatGPT's answer was selected. In one of them, the asker even selected an answer from someone who is participating in this thread and awarded them points. If that user had filtered this question out, they would have missed the chance of posting a great answer as they did and earning the points.
The concerns of those participating in this thread are acknowledged and taken seriously (let's not confuse "being heard" with "get what I want"), but there are other users of KudoZ that do not participate in forum discussions, but who are showing interest in this supporting integration. Usage patterns and private feedback of these users are also being taken into account.
Thanks again for your feedback. I'll post back here once the "This is an AI-generated answer" legend is available on the old interface (also the ability to see only questions with AI-generated answers for those who want to keep track of what is being done and would like to suggest improvements).
Lucia
The translation and interpretation of your message is:
"I read your messages about your disgust towards AI, but we don't care. We'll apply it anyway".
And that's quite disappointing from Proz, who put a big dagger in our profession. Those other users of KudoZ that do not participate in forum discussions, but who are showing interest in this supporting integration are free to show their interest in that "integration" in other spaces, without damaging the Kudoz section.
Please remove it.
| | | | | Pages in topic: < [1 2 3 4 5 6] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Introducing AI to KudoZ: human expertise and AI combined in support of the most helpful answer | Protemos translation business management system | Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!
The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.
More info » |
| | Wordfast Pro | Translation Memory Software for Any Platform
Exclusive discount for ProZ.com users!
Save over 13% when purchasing Wordfast Pro through ProZ.com. Wordfast is the world's #1 provider of platform-independent Translation Memory software. Consistently ranked the most user-friendly and highest value
Buy now! » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
